The design of CMOS image sensors is going through an accelerated phase of development to serve a variety of applications.
Around 540 million years ago, a rapid evolution of living things, known as the Cambrian explosion, led to diversification into various species. One of the reasons for this diversification was the development of various sensory organs, and the creation of the eye was the most important event in this process. A similar phenomenon appeared with the electronic devices we use. As digitalization continues, cameras play the role of eyes in mobile devices and CMOS image sensors (CIS) capture images like retinas.
Through CIS technology, we could easily process, reproduce, and save huge amounts of image information. As a result, mobile devices had to process an abundant amount of data, which again drove an explosive increase in the capacity and performance of application processors (APs) or memories that play the role of the brain. In addition, the camera gained more importance from the perspective of users, which led to a diversification of mobile devices.
The changes that occurred during the Cambrian period have been mirrored in our everyday life through electronic devices. And the recent COVID-19 pandemic, which opened the ‘contact-free’ era, has accelerated diversification and provided a burst of growth in electronic devices. The pandemic will end soon, but the trend will continue.
Like the eye
The biggest role of CIS is to accurately reproduce the world as we see it through our eyes. We want it to have a level of resolution similar to the human eye, and it has to see well in dark and bright environments, while also recognizing moving objects at high-speed.
Figure 1 demonstrates the basic pixel (PX) structure and operation characteristics of CIS. Light reflected off of an object passes through the optical system and into the photodiode, and when the photon energy of light exceeds the semiconductor’s bandgap energy, it creates an e-/h+ pair. Accumulating and reading this signal enables the formation of a 2D image depending on the intensity of the light. The bandgap energy of silicon is 1.1 eV and this energy can cover the entire visible spectrum of the human eye.
Figure 1 This diagram highlights basic pixel structure and operational characteristics of CIS. Source: SK hynix
Seeing well in dark conditions requires amplifying signals from a small amount of light, while suppressing non-light signals (noise) as much as possible. In addition, seeing in bright conditions requires receiving a large amount of light and distinguishing it well. These characteristics are quantified by indicators such as signal-to-noise ratio (SNR) and dynamic range (DR).
In the aspect of low-light SNR, there has been a lot of effort to amplify signals and reduce unwanted noise. Today, we are continuously improving these characteristics to the level of 5 lux, which is quite a dark environment. In terms of DR, while the intra-scene and inter-scene DR of the human eye is typically 120 dB and 180 dB, respectively, the intra-scene and inter-scene DR of the current smartphone is 70 dB and 120 dB, respectively, and constant improvement is being made.
The most important factors in affecting these characteristics are PX size and resolution. For the higher resolution of CIS, the PX size must be smaller. To implement CIS on a smaller area with the same resolution, the PX size also must be smaller. The key factor is to maintain the above characteristics at the same level despite shrinking the PX size.
How long will the competition for PX size and resolution continue? The resolution of the human eye is 576 megapixels around the center of a static image and about 8 megapixels on a moving image. CIS has continued to evolve to catch up to this level. The speed of development has somewhat slowed down around 1.12 μm in PX size and 13 megapixels in resolution, but the introduction of Quad technology—merging 2×2 pixels with the same color filter—once again accelerated the speed of shrinking PX size rapidly to the 0.7-μm range and 64 megapixels in resolution.
Now, with Nona (3×3) and QxQ (4×4) technologies, PX size has continued to evolve to the 0.6X-μm range. In turn, PX shrinking technology led to the recent release of a 108 megapixel-resolution sensor, with increasing expectation that a 200-megapixel camera resolution will soon be launched.
We have caught up a lot. Videos do not require high resolution, which is why the above-mentioned pixel-binding technologies were actively adopted. With this, cameras can now support 4K (4,000×2,000: 8 megapixels) video recording at 60 fps without interruption. Pixel-binding technology enables cameras to maintain the characteristics of large pixels in videos, delivering superior low-light sensitivity and DR. In the future, value added features to customers such as ultra-low light, dynamic range extension technology, and quick auto focus are expected to be added to video capturing.
Reducing area, including shrinking PX size is the fate of all semiconductor devices. There has been a great effort in terms of devices and processes to maintain the same characteristics on a smaller area. Such technologies include doping optimization and vertical transfer gate to enhance full well capacity (FWC) while maintaining charge transfer efficiency, source follower engineering, and various design technologies to reduce noise. Then there are color filter isolation and deep trench isolation technologies to minimize interference between adjacent pixels. Finally, engineers can use a thicker epi layer to enhance PX sensitivity or apply various color filter related technologies.
Through the aforementioned technologies, in terms of seeing images, semiconductors are now nearly-equal to the performance of the eyes of living creatures. However, in terms of energy efficiency, there are still improvements to be made. We are witnessing a trend developing in low-power technologies, such as turning on when required from standby mode running on minimal power—so called always-on mode—or optimizing power through compressive sensing methods.
Beyond the eye
CIS is further expanding its application area, and one representative is depth sensing technology. The human eye can sense distance using binocular disparity, and in the early days, CIS was developed to apply the same method using two cameras. However, due to the limitations in accuracy, distance expandability, and the requirement of minimum distance between the two cameras, there are now different efforts being made.
The method is called time of flight (ToF), which measures the distance based on the time difference of light returning after being reflected off an object. ToF can be categorized into two types: direct (dToF) and indirect (iToF). Each of the two methods has pros and cons derived from operation principles.
iToF sensing is limited in measurable distance range caused by signal diminishing in long distance based on its operation principle of analog charge accumulation. On the other hand, dToF sensing is limited in resolution due to the challenge of the size of the cell that detects even one single-photon avalanche diode (SPAD) and the need to stack readout circuits in each cell. Due to these strengths and weaknesses, the two methods are used in specific applications where their strengths can be realized and leveraged.
CIS is still seeking ways to contribute to enriching human life by expanding application fields while leveraging ultraviolet, infrared, near-infrared, and short-wave infrared based on the wide spectrum of light. These wavelengths will bring opportunities to replace the shortcomings of silicon with alternative materials such as Ge, InGaAs, and InP. In addition, multi- and hyper-spectral imaging, or polarization sensors have also begun to contribute.
Artificial intelligence (AI), which is being widely adopted across multiple industries, is also providing benefits to this field. CIS is now expanding applications to object recognition and security. However, the advent of cameras in many aspects of society leads to concerns around privacy protection. The key is transmitting data required to ensure safety and security to servers, while preventing other data from being exposed. Therefore, AI functions will migrate to edge devices, allowing us to achieve reduced data transmissions between IoT devices and power saving. The use of IR data and the emergence of event-driven sensors are also in line with this direction.
Along with these applications, a range of new technologies are emerging. That includes development of new wearable devices such as head-mounted displays (HMD) and AR/VR glasses and various sensors applied to applications like autonomous vehicles, robots, and drones.
This article was originally published on EDN.
Kangbong Seo is head of Future Innovation Technology at SK hynix.