Embedded Vision Summit: ML perception on tiny FPGAs

Article By : Nitin Dahad

At the Embedded Vision Summit, Lattice Semiconductor will demonstrate how tiny low power FPGAs can enable multiple image processing functions...

How much artificial intelligence can you implement on an FPGA using just milliwatts of power? And how quickly can you implement efficient artificial intelligence (AI) inferencing in smart internet of things (IoT) devices? These are the questions Lattice Semiconductor hopes to address at this month’s virtual Embedded Vision Summit from 15-25 September 2020.

In a presentation entitled “Machine-Learning-Based Perception on a Tiny, Low-Power FPGA”, Hoon Choi, an R&D fellow at Lattice Semiconductor, will demonstrate how these can enable hand gesture classification, human detection and counting, local face identification, location feature extraction, front facing human detection and shoulder surfing detection, among others.

Choi will describe Lattice Semiconductor’s compact processing engine structure that fits into fewer than 5K FPGA look-up tables yet can support networks of various sizes. He describes how networks were selected and the optimizations used to make them suitable for low-power and low-cost edge applications. He also talks about how to leverage on-the-fly self-reconfiguration capability of FPGAs to enable running a sequence of processing engines and neural networks in a single FPGA.

Lattice Semiconductor recently announced the second device in its CrossLink-NX family of embedded vision and processing FPGAs. The new device, CrossLink-NX-17, features 17K logic cells, compared to the previous CrossLink-NX-40 with 39K logic cells, which has been shipping in production quantities since 2019. The CrossLink-NX family was designed using the Lattice Nexus platform, a low power FPGA platform using a 28 nm FD-SOI manufacturing process and featuring a Lattice-designed FPGA fabric architecture optimized for low power operation in a small form factor.

Lattice Semiconductor Image_Sensor_Processing
Image sensor processing with the CrossLink-NX FPGA: bridge multiple sensors to processor interface and offload ISP functionality from processor. (Image: Lattice Semiconductor)

Demonstrations at the Embedded Vision Summit will include a CrossLink-NX FPGA performing human presence detection and counting, while another CrossLink-NX board will illustrate the device’s camera aggregation capabilities for embedded vision applications. Additionally, a Lattice sensAI user awareness kit will show a tiny, ultra-low power iCE40 UltraPlus FPGA using the sensAI algorithm to detect and analyze human activity at a computer workstation.

For the partner demonstrations, Lattice will team with Helion-Vision to bring its IONOS ISP (image signal processing) IP portfolio to the Lattice CrossLink-NX and ECP5 FPGAs. This demonstration will show the quality of Helion-Vision’s image processing algorithms, along with optional features like image overlay. Ignitarium Technology Solutions, experts in applied AI technology, will show how efficient ML/AI algorithms in the Lattice ECP5 FPGA can perform complex tasks, such as identifying defective products as they move along a conveyor belt.

According to a report by BCC Research, “Computer Vision and Machine Vision in Everyday Life,” the global market for computer and machine vision was worth $14.9 billion in 2019 and is set to grow to $26 billion by 2024. Lattice said it helps developers address this demand for embedded and smart vision applications with a variety of low-power FPGAs and solutions stacks to enable quick and easy implementation of applications like video signal bridging, aggregation and splitting, image processing, and the AI/ML inferencing used to train smart vision models.

Peiju Chiang, product marketing manager at Lattice, said, “With the CrossLink-NX-17, Lattice gives developers one more hardware power and performance option to choose from as they design their vision systems.” He added that its mVision solutions stack further accelerates and simplifies vision system development by providing modular hardware development boards, featuring Lattice FPGAs like the CrossLink-NX, its Radiant 2.1 design software, embedded vision IP, and reference designs needed to implement popular embedded vision applications.

Leave a comment