Optimizing thermal imaging designs
Article By : Bobby John
Understanding the differences between thermal and optical imaging can help developers optimize their designs.
Thermal imaging is used in a wide range of applications, from the manufacturing and processing of industrial products to security and surveillance. Because the wavelengths being measured by thermal cameras are larger than those measured in optical imaging, developers of thermal imaging applications need to approach design differently than is used in traditional vision applications. By understanding the differences between thermal and optical imaging, developers can optimize their designs to utilize the right kind of external memory that results in smaller systems, lower complexity, reduced power consumption, and ultimately lower system cost.
The Infrared Spectrum
The human eye is only capable of capturing a very small portion of the greater electromagnetic spectrum called the visible spectrum. Outside of this region lie other spectrums such as x-ray, ultraviolet (UV), infrared (IR), and microwave whose frequency and wavelength make them indiscernible to the human eye.
Of special importance in this discussion is the IR spectrum. The IR spectrum provides a means to detect and measure the heat generated by an object. This is referred to as the “heat signature”. The hotter an object is, the more infrared radiation it produces.
Thermal cameras are instruments that can capture infrared radiation and convert it to an image that we can then see with our eyes. Though infrared imaging was originally developed to locate enemy targets at night, thermal imaging is now used in many different types of applications including:
- Preventive early fire detection and plant condition monitoring
- Manufacture and processing in factory automation (Industry 4.0)
- Energy conservation such as identifying where a house might be losing heat through door and window cracks.
- Night-time surveillance, such as perimeter protection
- Weather tracking such as of storms and hurricanes
- Diagnosis of different disorders and diseases.
- Inspection of vehicles (e.g., airplanes and trains)
- Wildlife conservation, agriculture, animal husbandry
- Disaster rescue and recovery
The list of applications making use of thermometry continues to grow. As companies invest more in research and development, thermal cameras will only get better and less expensive, thus finding their way into even more applications, from recreation to research.
Thermal cameras are available in choice of sensors, fields of view, frame rates and physical configurations. A thermal camera is made up of a mechanical housing with lens, infrared sensor, and processing electronics consisting of the image processor, FPGA, memory, communication, and display electronics. The lens focuses infrared energy onto the sensor, which measures the heat signature of any objects in the environments.
Thermal sensors come in a variety of pixel configurations, from 80 × 60 to 1280 × 1024 pixels or more. Note that these resolutions are low in comparison to visible light imagers. Because thermal detectors need to sense energy that has much larger wavelengths than visible light, each sensor element has to be significantly larger as well. Consider that standard consumer cameras have a pixel size of around 1.7µm while industrial machine vision cameras have pixel sizes ranging from 4.6µm to 6.5µm with a larger light-active surface to obtain a better signal. Thermal cameras have even larger sensors, with a pixel size of 25µm. As a result, a thermal camera usually has much lower resolution (i.e., fewer overall pixels) than visible sensors of the same mechanical size.
Note that while larger pixel size reduces resolution, it also means that heat sensed by an infrared camera can be very precisely measured. This is important for a large variety of applications. For example, some thermal cameras can detect tiny differences in heat—as small as 0.01°C—and display them as shades of grey or using different color palettes.
Memory Challenges
The FPGA within a thermal camera filters and processes the signals generated by its sensors and detectors. Often the RAM block within the FPGA is insufficient for storing and processing the data. The system will have to depend on off-chip image memory for tasks such as running algorithms, displaying data, and buffering communications. Expansion memory also provides the additional benefit of allowing the design to be scalable to meet expanding density requirements.
Traditionally, OEMs have used DRAM for off-chip storage utilizing a DDR interface. However, given the low image resolution requirements of thermal imaging, off-chip memory requirements are substantially lower than is required by optical cameras. As such, a high-density DRAM might be overkill and increase product cost without providing any real benefit. DRAMs also typically require more than 30 pins for data transfer. These pins add to system overhead in terms of additional signal routing and requiring additional PCB layers to run these signal traces. Furthermore, since DRAM is volatile, cells need to be periodically refreshed to preserve data. Thus, using too large a DRAM means higher power consumption, directly impacting operating life for battery powered thermal imaging applications.
To address the memory challenges of DRAM, camera OEMs are using alternative memory technologies such as HyperRAM memory. HyperRAM is based on the DRAM architecture and includes in-built self-refresh circuitry. Requiring an active current of only 25mA, HyperRAM power consumption is a fraction of that compared to DRAMs (see Table 1), making it power efficient enough for portable applications.
Table: Comparison of HyperRAM vs Single Data Rate (SDR) DRAM. [*Note: comparison is using a 64Mb device as a base.] (Source: Infineon Technologies)
The HyperBus Memory Interface and Protocol provides equivalent throughout to DDR – 400MBps – while requiring only 12 pins for data transfer. Rather than having to implement a costly DDR DRAM memory controller, a gate-count efficient HyperBus Memory Controller can be implemented in soft IP in the FPGA, making this an optimal and efficient approach to off-chip expansion memory (see Figure 1).
click for full size image
Figure 1: (Left) A camera using external DDR SDRAM and NOR Flash requires two memory buses totalling 41 pins that increase the PCB layers to six or more. (Right) A camera using HyperRAM and HyperFlash for external memory can communicate over a single bus of 13 pins, and requires only two to four PCB layers. (Source: Infineon Technologies)
Most camera designs also have a requirement for external NOR Flash to store parameters and other important information that needs to be retained when the power is turned off (battery powered) or there is power failure. With standard NOR Flash, another 10 pins will be required for the bus interface, bringing the pin total to 41. As an alternative to NOR Flash, OEMs can use HyperFlash memory.
HyperFlash is NOR Flash that utilizes the HyperBus interface. This enables systems to utilize the same bus for interfacing with both HyperRAM and HyperFlash devices to reduce the overall pin count even further. In this case, only 13 pins total would be required for interfacing: 12 pins for data transfer and 1 additional pin for use as a chip select. Compare this to the 41+ pins that might be required for separate DDR DRAM and QSPI NOR Flash devices.
Note that HyperRAM expansion memory can also be used in industrial machine vision applications as an alternative to DRAM for the image memory. With low-pin count packaging, HyperRAM is available in densities ranging from 64Mb to 512Mb supporting both HyperBus and Octal xSPI JEDEC compliant interfaces. HyperBus is supported by an ecosystem of partners, and the HyperBus memory controller is also available as RTL IP for implementing the controller in an FPGA.
Developers of thermal cameras must address challenges that are different than what designers of optical cameras face. By selecting an external memory technology that matches the requirements of a thermal imaging system, OEMs can simplify their signal tracing, reduce the number of PCB layers needed, lower overall system cost, and reduce power consumption to improve operating life.
This article was originally published on Embedded.
Bobby John is Senior Product Marketing Manager in the Memory Solutions business at Infineon Technologies and has over 13 years of experience across various roles in the Semiconductor Industry.
