feature article
Subscribe Now

Samsung’s 50-Mpixel GN2 Image Sensor Sports Pro Camera Features

Could this Sensor Kill Standalone Cameras Completely?

In August, at the Hot Chips 33 conference, held online this year thanks to COVID-19, Sooki Yoon described the Samsung 50-Mpixel GN2 image sensor, which the company announced earlier this year. This sensor is destined to be used in mobile phones such as the Xiaomi Mi 11 Ultra/Pro, according to Yoon’s presentation, but it creates a camera with features that shame those of most compact, mirrorless, and dSLR cameras.

Camera phones have been systematically wiping out the photographic enthusiast camera market as their mobile phone cameras accrete better and better features. The Samsung GN2 sensor may be good enough to put the final stake in the heart of consumer devices that can “only” function as cameras – you know, like cameras.

The Samsung GN2 image sensor is a 2-die sandwich consisting of an image sensor die with 8160×6144 pixels and a die that holds the Analog-to-Digital Converter (ADC) and the image-processing digital circuitry. The pixel die holds the 50 million 1.4 μm pixels and therefore uses larger lithographic geometries to image the circuits on the die. The die containing the ADC and image-processing circuitry uses 28 nm lithography, which is the current economic sweet spot for chip making.

The GN2 sensor has numerous outstanding features including:

  •         50-Mpixel resolution
  •         Advanced DPAF (Dual-Pixel auto Focus) using diagonally sliced phase-detecting pixels
  •         Smart ISO Pro for wide dynamic range
  •         Staggered HDR (High Dynamic Range)
  •         Significant power reduction, thanks to reduced ADC operating voltage

Each of these features deserves its own elaboration, so here it is:

50-Mpixel resolution

Image sensor resolution is always at war with pixel dynamic range. To get more resolution, you need more pixels. Put more pixels on the same size chip and each pixel must become smaller. But the charge wells of smaller pixels hold fewer electrons, which results in noisy shadows.

Samsung’s solution to the competing design goals of more pixel resolution and low image noise is to put the 50 Mpixels on the GN2 sensor on a larger piece of silicon. Figure 1 illustrates the evolution of Samsung mobile phone image sensors culminating in the GN2.

Fig 1: The Samsung GN2 mobile image sensor achieves high resolution with low noise by using an enlarged sensor die. (Image Credit: Samsung)

As Fig 1 shows, the Samsung GN2 image sensor achieves high resolution with low noise by placing larger pixels with larger charge wells on a larger silicon die. Previous Samsung image sensors employed 0.8 and 1.2 μm pixels, while the GN2 image sensor employs 1.4 μm pixels. The result, shown in Fig 2, is greatly increased sensitivity thanks to a 33 percent boost in the maximum number of electrons that can be stored in the pixel charge well, which results in more dynamic range with less visible noise in the image shadows.

Fig 2: The Samsung GN2 image sensor’s 1.4 μm pixel stores 33 percent more electrons than a 0.8 μm pixel, which results in more dynamic range and lower noise in the image shadows. (Image Credit: Samsung)

Advanced DPAF

The top focusing algorithm for cameras at the moment is dual-pixel autofocus (DPAF), which uses a split pixel to create a double image on one imaging pixel. Each imaging pixel consists of two photodiodes that can be used for imaging and for focusing. Fig 3 illustrates how this technique works.

Fig 3: Dual-Pixel Auto Focus (DPAF) projects left and right images on a split pixel and detects when an image is in focus by sensing when the phases of the left and right images match on the split pixel readout. (Image Credit: Samsung)

The DPAF focusing technique projects left and right images on a pixel’s two photodiodes and detects when an image is in focus by sensing when the phase outputs of the left and right photodiodes match during a split-pixel readout. During imaging, the outputs of the left and right photodiodes are combined to produce the image.

DPAF isn’t new. For example, Canon has been using DPAF on its cameras starting with the 70D dSLR, introduced in 2013. However, DPAF can have an Achilles’ heel: if the the two photodiodes in the phase-sensing pixel are optically isolated with a vertical optical wall, then the pixel cannot detect the phase shift of horizontal lines because horizontal lines look the same (have the same phase) when shifted right or left. Samsung’s ingenious solution is to optically split some of the GN2 sensor’s pixels – the green ones – diagonally instead of vertically. To be clear, the green photodiodes are vertically split electrically and diagonally split optically, as shown in Fig 4.

 

Fig 4: Earlier dual-pixel phase detection directed light to the left and right phase-detecting pixel photodiodes using vertical Deep Trench Isolation (DTI). The Samsung GN2 sensor uses a diagonal DTI structure to allow the two photodiodes in a green pixel to detect and focus on both horizontal and vertical lines. (Image Credit: Samsung)

Diagonal optical isolation allows each green photodiode to respond to phase differences for both horizontal and vertical lines, which gives the camera better DPAF capabilities. Fig 5 illustrates the difference between vertically isolated and diagonally isolated phase-detection subpixels. In the image on the left, the vertically isolated DPAF sensor cannot focus in any zone because there are only horizontal lines in the zone. In the image on the right in Fig 5, the sensor can focus on all of the horizontal lines in each zone.

Fig 5: Vertically isolated phase-detection subpixels cannot focus on horizontal lines because shifting these lines right or left does not shift the phase of the resulting signal in the photodiode. A diagonally isolated, phase-detecting pixel can detect horizontal lines. (Image Credit: Samsung)

Smart ISO Pro

Each group of four pixels in the Samsung GN2 sensor has a separate dual gain control switch to adjust the ISO sensitivity of the pixel group. For brightly lit images, the sensor turns the gain of the pixels down to match the full voltage span from the pixel’s image signal to the full input range of the pixel ADC (Low ISO mode). For low light, the sensor boosts the gain of the pixel group so that the full input range of the ADC covers just the low end of the luminance range (High ISO mode), as shown in Fig 6.

Fig 6: The Samsung GN2 image sensor can operate in High ISO mode for dimly lit scenes, to achieve lower noise, and switches to Low ISO mode for brightly lit scenes. (Image Credit: Samsung) 

Staggered HDR

High Dynamic Range (HDR) is a relatively new imaging technique that “appears” to extend the camera’s dynamic luminance range. It’s a compression technique that merges two or more images taken at different sensor sensitivities to accommodate both the brightly lit and dimly lit portions of an image. It helps to bring out details in the shadows of high-contrast scenes. Because HDR techniques use multiple images of the same scene to achieve the effect, HDR can result in blurry scenes if the separate images don’t overlay perfectly. Image stretching algorithms can fix some of the blur, but the best way to reduce blur is to reduce the time between the captures of the multiple images.

Staggered HDR is a way to reduce blur by taking three different exposures in the time normally used to take one image. Samsung says this technique makes use of a rolling shutter, which allows the image sensor to take a long exposure, followed by a medium exposure, and then a short exposure, for dim, moderate, and bright imaging respectively, and then output those three images with some overlap. Using the rolling shutter, the sensor first takes the long exposure and starts to output the exposure a row at a time, with all pixel voltages in a row transferring simultaneously from the pixel row to the ADC.

Before all rows of the long exposure have been fully output, the sensor takes the medium exposure and starts to output it before the long exposure has been fully output. It then does the same for the short exposure. The sensor uses virtual MIPI channels to output the three exposures with some overlap, and the GNR sensor’s image-processing die weaves the images together to produce the HDR image. If this all seems confusing, perhaps Fig 7 can clear up the sequence of events.

Fig 7: To reduce blur in Staggered HDR mode, the Samsung GN2 sensor takes a long exposure for dimly lit areas and starts to output that image on one virtual MIPI channel (VC0). Before that long-exposure image is fully output, the sensor takes a medium-exposure image and starts to output that image on a second MIPI virtual channel (VC1). It then takes a short exposure for brightly lit areas and outputs that image on a third MIPI virtual channel (VC2). (Image Credit: Samsung)

As Fig 7 shows, the sensor can simultaneously send the autofocus information (marked “AF” in the figure) generated by the DPAF feature on a fourth MIPI virtual channel (VC3).

Significant power reduction

Mobile phones constantly battle Battery’s Law: battery energy capacity does not grow nearly as fast as do silicon-driven capabilities. Consequently, there’s a continuing need to cut power consumption wherever possible. The designers of the Samsung GN2 image sensor attacked one obvious power hog: the high-speed ADC used to convert analog pixel voltages to digital representation. Dropping the ADC’s supply voltage from 2.8 V to 2.2 V cut the ADC’s power consumption by more than 20%, from 311 to 244 mW.

However, merely cutting the supply voltage would have reduced the ADC’s input range to the point where it could not accept the full range of pixel image voltages from the sensor. To combat this problem, Samsung’s engineers added a -0.6 V substrate back bias to the pixel image sensor die, effectively level shifting the sensor’s output range, and they developed a low-threshold-voltage transistor design for use in the ADC circuitry on the 28 nm die. The combination of these two innovations reduced operating power while preserving the full sensor exposure range.

Yoon discussed other innovations designed into the Samsung GN2 image sensor during his presentation at Hot Chips 33, but the features listed above were the ones that really piqued my interest, as a design engineer and as a long-time camera enthusiast who owns several digital cameras. Samsung’s engineers packed a massive amount of technological innovation into this sensor, and I believe it really raises the capability bar for all cameras, in a mobile phone or otherwise.

2 thoughts on “Samsung’s 50-Mpixel GN2 Image Sensor Sports Pro Camera Features”

Leave a Reply

featured blogs
Nov 30, 2021
We live in a world where the idea of usability is to make products easy to use, make things easily accessible, and visually appealing. It's our constant endeavor to improve the usability of our... [[ Click on the title to access the full blog on the Cadence Community si...
Nov 29, 2021
Tell me if you've heard this before, but I'm looking for a Nordic word that has a sufficiently amorphous gestalt to make it confusing to explain in Norwegian....
Nov 29, 2021
Lean how virtual electronic control units (ECUs) accelerate automotive design and enable advanced driver-assistance systems (ADAS) for connected vehicles. The post From Road to PC: Accelerating Intelligent Software Growth with Virtual ECUs appeared first on From Silicon To S...
Nov 8, 2021
Intel® FPGA Technology Day (IFTD) is a free four-day event that will be hosted virtually across the globe in North America, China, Japan, EMEA, and Asia Pacific from December 6-9, 2021. The theme of IFTD 2021 is 'Accelerating a Smart and Connected World.' This virtual event ...

featured paper

Watch PCIe Gen5 Interface Demo Running on a Speedster®7t FPGA from Achronix

Sponsored by Achronix

PCIe Gen5 is the most advanced PCIe specification available today, providing data link capable of a 32 GT/s for next-generation systems. You can start to design your PCIe Gen5 system today using the Achronix Speedster7t FPGA. This demonstration shows a successful PCIe Gen5 link between a Lecroy PCIe exerciser and a Speedster7t FPGA. The Speedster7t family is one of the first FPGAs available now that natively supports the PCIe Gen5 specification.

Contact Achronix for a Demonstration of Speedster7t FPGA

featured paper

3D-IC Design Challenges and Requirements

Sponsored by Cadence Design Systems

3D-ICs are expected to have a broad impact on networking, graphics, AI/ML, and high-performance computing. While there’s interest in 3D-ICs, it’s still in its early phases. Standard definitions are lacking, the supply chain ecosystem is in flux, and design, analysis, verification, and test challenges need to be resolved. Read this white paper to learn about 3D integration and packaging of multiple stacked dies, design challenges, ecosystem requirements, and needed solutions.

Click here to read more

featured chalk talk

Accelerating Innovation at the Edge with Xilinx Adaptive System on Modules

Sponsored by Xilinx

The combination of system-on-module technology with advanced SoCs with programmable logic offer the ultimate in functionality, performance, flexibility, power efficiency, and ease of use. In this episode of Chalk Talk, Amelia Dalton chats with Karan Kantharia of Xilinx about the new Kira SOM, and how it enables faster time-to-deployment versus conventional component-based design.

Click here for more information about Kria Adaptive System-on-Modules