CES 2021: 5nm AI chip enables 8K video processing in just 2W
Article By : Nitin Dahad
Using CES 2021 as the launch platform, Ambarella has announced its 5th generation AI vision processor SoC...
Using CES 2021 as the launch platform, Ambarella has announced its fifth generation artificial intelligence (AI) vision processor system on chip (SoC) which it said sets a new standard for power efficiency. Its CV5 AI vision CVflow processor fabricated in an advanced Samsung 5nm process enables encoding of full 8K video at 30 frames per second in under 2 Watts, which is significant for intelligent camera systems in automotive, consumer (including drones) and robotic cameras.
The new CV5 combines Ambarella’s CVflow AI engine with dual Arm A76 CPUs to provide the performance necessary for a wide range of AI-based algorithms. Enabling input from up to 14 cameras, the image signal processor (ISP) can simultaneously process images for both human viewing and machine processing. In a briefing with embedded.com, Chris Day, vice president of marketing and business development at Ambarella, said, “This is our fifth generation of image processors, with separate processing for video and AI. It’s the first 5nm chip, enabling low power consumption which is critical for many applications, and especially in sports cameras.”
He added, “We work closely with the Samsung fab and have one of the best VLSI design teams in the industry. This means we have good early access to the process libraries as a result of our close working relationship with Samsung.”
In automotive video telematics applications, CV5 provides the performance necessary to encode multiple video streams from front advanced driver assistance systems (ADAS), driver monitoring, cabin monitoring, and side-view cameras. Its CVflow AI engine can simultaneously run ADAS algorithms such as lane departure, and forward collision warning as well as driver monitoring algorithms such as drowsy driver detection. The combination of high-resolution video capture with advanced AI processing enables ADAS cameras to recognize images over long distances and with high accuracy.
Day said that initial interest is in the automotive market, as this segment accounts for about 20% of Ambarella’s business.
The low power consumption of the CV5 SoC also makes it suitable for next generation high resolution and high frame-rate action, 360°, and virtual reality (VR) cameras, offering 8K recording and playback in very small form factor designs. In particular, it also has features such as on-chip image stabilization and stitching suitable for VR cameras.
In robotic and drone applications, CV5’s CVflow AI engine can accelerate simultaneous localization and mapping (SLAM), path planning, and obstacle detection, and avoidance algorithms for navigation and autonomous operation. For drone-based aerial videography and cinematography, CV5 can simultaneously perform flight control and navigation functions while recording up to 8K resolution video recording at 60 frames per second.
Day commented there are two areas in the drone market that the CV5 is particularly good for: one is in improving videography, but the other is in adding computer vision and AI for things like object detection.
Across all the different applications, Day said the key benefit of the CV5 is the ability to add 8K recording across all markets. He also said that this is the first time the company has included the PCIe interface on chip, opening up connectivity options. While the initial focus is on automotive, sports cameras, drones and VR cameras, he expects to have versions of CV5 for security and surveillance later this year
The new SoC shares common SDK and computer vision (CV) tools with Ambarella’s other CVflow SoC families, simplifying development of cameras with multiple price and performance options. A complete set of CV tools helps customers port their own neural networks onto CV5, including a compiler, debugger, and support for industry-standard machine learning frameworks, such as PyTorch, ONNX, Caffe and TensorFlow, as well as extensive guidelines for convolutional neural network (CNN) performance optimization.
The main features of the CV5 SoC include:
- CVflow architecture with DNN support
- Dual-core Arm Cortex-A76 up to 1.6 GHz with NEON DSP extensions and FPU
- High-speed SLVS-EC, MIPI-CSI (C/D PHY) interfaces, connecting up to 14 cameras
- Multi-channel ISP with up to 8KP60 throughput
- Native support for RGGB, RCCB, RCCC, RGB-IR, and monochrome sensor formats
- Multi-exposure high dynamic range (HDR) processing
- Real-time, hardware-accelerated fisheye dewarping and lens distortion correction (LDC)
- AVC and HEVC encoding with up to 8KP60 throughput and multiple stream support
- Rich set of interfaces includes 4-lane PCIe, CAN FD, Gigabit Ethernet, USB 3.1 host and device, three SD card controllers, MIPI DSI/CSI-2 and HDMI video outputs
- LPDDR4x / LPDDR5 / LPDDR5x support, 64-bit data bus and up to 32 GB capacity
- Security features including secure boot with TrustZone and secure memory, TRNG, OTP, DRAM scrambling and virtualization
- 5 nm process technology
- 16×16 FBGA package with 0.5 mm ball pitch