feature article
Subscribe Now

Samsung’s 50-Mpixel GN2 Image Sensor Sports Pro Camera Features

Could this Sensor Kill Standalone Cameras Completely?

In August, at the Hot Chips 33 conference, held online this year thanks to COVID-19, Sooki Yoon described the Samsung 50-Mpixel GN2 image sensor, which the company announced earlier this year. This sensor is destined to be used in mobile phones such as the Xiaomi Mi 11 Ultra/Pro, according to Yoon’s presentation, but it creates a camera with features that shame those of most compact, mirrorless, and dSLR cameras.

Camera phones have been systematically wiping out the photographic enthusiast camera market as their mobile phone cameras accrete better and better features. The Samsung GN2 sensor may be good enough to put the final stake in the heart of consumer devices that can “only” function as cameras – you know, like cameras.

The Samsung GN2 image sensor is a 2-die sandwich consisting of an image sensor die with 8160×6144 pixels and a die that holds the Analog-to-Digital Converter (ADC) and the image-processing digital circuitry. The pixel die holds the 50 million 1.4 μm pixels and therefore uses larger lithographic geometries to image the circuits on the die. The die containing the ADC and image-processing circuitry uses 28 nm lithography, which is the current economic sweet spot for chip making.

The GN2 sensor has numerous outstanding features including:

  •         50-Mpixel resolution
  •         Advanced DPAF (Dual-Pixel auto Focus) using diagonally sliced phase-detecting pixels
  •         Smart ISO Pro for wide dynamic range
  •         Staggered HDR (High Dynamic Range)
  •         Significant power reduction, thanks to reduced ADC operating voltage

Each of these features deserves its own elaboration, so here it is:

50-Mpixel resolution

Image sensor resolution is always at war with pixel dynamic range. To get more resolution, you need more pixels. Put more pixels on the same size chip and each pixel must become smaller. But the charge wells of smaller pixels hold fewer electrons, which results in noisy shadows.

Samsung’s solution to the competing design goals of more pixel resolution and low image noise is to put the 50 Mpixels on the GN2 sensor on a larger piece of silicon. Figure 1 illustrates the evolution of Samsung mobile phone image sensors culminating in the GN2.

Fig 1: The Samsung GN2 mobile image sensor achieves high resolution with low noise by using an enlarged sensor die. (Image Credit: Samsung)

As Fig 1 shows, the Samsung GN2 image sensor achieves high resolution with low noise by placing larger pixels with larger charge wells on a larger silicon die. Previous Samsung image sensors employed 0.8 and 1.2 μm pixels, while the GN2 image sensor employs 1.4 μm pixels. The result, shown in Fig 2, is greatly increased sensitivity thanks to a 33 percent boost in the maximum number of electrons that can be stored in the pixel charge well, which results in more dynamic range with less visible noise in the image shadows.

Fig 2: The Samsung GN2 image sensor’s 1.4 μm pixel stores 33 percent more electrons than a 0.8 μm pixel, which results in more dynamic range and lower noise in the image shadows. (Image Credit: Samsung)

Advanced DPAF

The top focusing algorithm for cameras at the moment is dual-pixel autofocus (DPAF), which uses a split pixel to create a double image on one imaging pixel. Each imaging pixel consists of two photodiodes that can be used for imaging and for focusing. Fig 3 illustrates how this technique works.

Fig 3: Dual-Pixel Auto Focus (DPAF) projects left and right images on a split pixel and detects when an image is in focus by sensing when the phases of the left and right images match on the split pixel readout. (Image Credit: Samsung)

The DPAF focusing technique projects left and right images on a pixel’s two photodiodes and detects when an image is in focus by sensing when the phase outputs of the left and right photodiodes match during a split-pixel readout. During imaging, the outputs of the left and right photodiodes are combined to produce the image.

DPAF isn’t new. For example, Canon has been using DPAF on its cameras starting with the 70D dSLR, introduced in 2013. However, DPAF can have an Achilles’ heel: if the the two photodiodes in the phase-sensing pixel are optically isolated with a vertical optical wall, then the pixel cannot detect the phase shift of horizontal lines because horizontal lines look the same (have the same phase) when shifted right or left. Samsung’s ingenious solution is to optically split some of the GN2 sensor’s pixels – the green ones – diagonally instead of vertically. To be clear, the green photodiodes are vertically split electrically and diagonally split optically, as shown in Fig 4.


Fig 4: Earlier dual-pixel phase detection directed light to the left and right phase-detecting pixel photodiodes using vertical Deep Trench Isolation (DTI). The Samsung GN2 sensor uses a diagonal DTI structure to allow the two photodiodes in a green pixel to detect and focus on both horizontal and vertical lines. (Image Credit: Samsung)

Diagonal optical isolation allows each green photodiode to respond to phase differences for both horizontal and vertical lines, which gives the camera better DPAF capabilities. Fig 5 illustrates the difference between vertically isolated and diagonally isolated phase-detection subpixels. In the image on the left, the vertically isolated DPAF sensor cannot focus in any zone because there are only horizontal lines in the zone. In the image on the right in Fig 5, the sensor can focus on all of the horizontal lines in each zone.

Fig 5: Vertically isolated phase-detection subpixels cannot focus on horizontal lines because shifting these lines right or left does not shift the phase of the resulting signal in the photodiode. A diagonally isolated, phase-detecting pixel can detect horizontal lines. (Image Credit: Samsung)

Smart ISO Pro

Each group of four pixels in the Samsung GN2 sensor has a separate dual gain control switch to adjust the ISO sensitivity of the pixel group. For brightly lit images, the sensor turns the gain of the pixels down to match the full voltage span from the pixel’s image signal to the full input range of the pixel ADC (Low ISO mode). For low light, the sensor boosts the gain of the pixel group so that the full input range of the ADC covers just the low end of the luminance range (High ISO mode), as shown in Fig 6.

Fig 6: The Samsung GN2 image sensor can operate in High ISO mode for dimly lit scenes, to achieve lower noise, and switches to Low ISO mode for brightly lit scenes. (Image Credit: Samsung) 

Staggered HDR

High Dynamic Range (HDR) is a relatively new imaging technique that “appears” to extend the camera’s dynamic luminance range. It’s a compression technique that merges two or more images taken at different sensor sensitivities to accommodate both the brightly lit and dimly lit portions of an image. It helps to bring out details in the shadows of high-contrast scenes. Because HDR techniques use multiple images of the same scene to achieve the effect, HDR can result in blurry scenes if the separate images don’t overlay perfectly. Image stretching algorithms can fix some of the blur, but the best way to reduce blur is to reduce the time between the captures of the multiple images.

Staggered HDR is a way to reduce blur by taking three different exposures in the time normally used to take one image. Samsung says this technique makes use of a rolling shutter, which allows the image sensor to take a long exposure, followed by a medium exposure, and then a short exposure, for dim, moderate, and bright imaging respectively, and then output those three images with some overlap. Using the rolling shutter, the sensor first takes the long exposure and starts to output the exposure a row at a time, with all pixel voltages in a row transferring simultaneously from the pixel row to the ADC.

Before all rows of the long exposure have been fully output, the sensor takes the medium exposure and starts to output it before the long exposure has been fully output. It then does the same for the short exposure. The sensor uses virtual MIPI channels to output the three exposures with some overlap, and the GNR sensor’s image-processing die weaves the images together to produce the HDR image. If this all seems confusing, perhaps Fig 7 can clear up the sequence of events.

Fig 7: To reduce blur in Staggered HDR mode, the Samsung GN2 sensor takes a long exposure for dimly lit areas and starts to output that image on one virtual MIPI channel (VC0). Before that long-exposure image is fully output, the sensor takes a medium-exposure image and starts to output that image on a second MIPI virtual channel (VC1). It then takes a short exposure for brightly lit areas and outputs that image on a third MIPI virtual channel (VC2). (Image Credit: Samsung)

As Fig 7 shows, the sensor can simultaneously send the autofocus information (marked “AF” in the figure) generated by the DPAF feature on a fourth MIPI virtual channel (VC3).

Significant power reduction

Mobile phones constantly battle Battery’s Law: battery energy capacity does not grow nearly as fast as do silicon-driven capabilities. Consequently, there’s a continuing need to cut power consumption wherever possible. The designers of the Samsung GN2 image sensor attacked one obvious power hog: the high-speed ADC used to convert analog pixel voltages to digital representation. Dropping the ADC’s supply voltage from 2.8 V to 2.2 V cut the ADC’s power consumption by more than 20%, from 311 to 244 mW.

However, merely cutting the supply voltage would have reduced the ADC’s input range to the point where it could not accept the full range of pixel image voltages from the sensor. To combat this problem, Samsung’s engineers added a -0.6 V substrate back bias to the pixel image sensor die, effectively level shifting the sensor’s output range, and they developed a low-threshold-voltage transistor design for use in the ADC circuitry on the 28 nm die. The combination of these two innovations reduced operating power while preserving the full sensor exposure range.

Yoon discussed other innovations designed into the Samsung GN2 image sensor during his presentation at Hot Chips 33, but the features listed above were the ones that really piqued my interest, as a design engineer and as a long-time camera enthusiast who owns several digital cameras. Samsung’s engineers packed a massive amount of technological innovation into this sensor, and I believe it really raises the capability bar for all cameras, in a mobile phone or otherwise.

2 thoughts on “Samsung’s 50-Mpixel GN2 Image Sensor Sports Pro Camera Features”

Leave a Reply

featured blogs
Sep 29, 2023
Our ultra-low-power SiWx917 Wi-Fi SoC with an integrated AI/ML accelerator simplifies Edge AI for IoT device makers. Accelerate your AIoT development....
Sep 29, 2023
Cadence has become a contributor-level member of the Automotive Working Group in the Universal Chiplet Interconnect Express (UCIe) Consortium. Last year, the Consortium ratified the UCIe specification, which was established to standardize a die-to-die interconnect for chiplet...
Sep 28, 2023
See how we set (and meet) our GHG emission reduction goals with the help of the Science Based Targets initiative (SBTi) as we expand our sustainable energy use.The post Synopsys Smart Future: Our Climate Actions to Reduce Greenhouse Gas Emissions appeared first on Chip Des...
Sep 27, 2023
On-device generative AI brings many exciting advantages, including cost, privacy, performance and personalization '“ offering significant enhancements in utility, productivity and entertainment with use cases across industries, from the commonplace to the creative....
Sep 21, 2023
Not knowing all the stuff I don't know didn't come easy. I've had to read a lot of books to get where I am....

featured video

TDK PowerHap Piezo Actuators for Ideal Haptic Feedback

Sponsored by TDK

The PowerHap product line features high acceleration and large forces in a very compact design, coupled with a short response time. TDK’s piezo actuators also offers good sensing functionality by using the inverse piezo effect. Typical applications for the include automotive displays, smartphones and tablet.

Click here for more information about PowerHap Piezo Actuators

featured paper

Intel's Chiplet Leadership Delivers Industry-Leading Capabilities at an Accelerated Pace

Sponsored by Intel

We're proud of our long history of rapid innovation in #FPGA development. With the help of Intel's Embedded Multi-Die Interconnect Bridge (EMIB), we’ve been able to advance our FPGAs at breakneck speed. In this blog, Intel’s Deepali Trehan charts the incredible history of our chiplet technology advancement from 2011 to today, and the many advantages of Intel's programmable logic devices, including the flexibility to combine a variety of IP from different process nodes and foundries, quicker time-to-market for new technologies and the ability to build higher-capacity semiconductors

To learn more about chiplet architecture in Intel FPGA devices visit: https://intel.ly/47JKL5h

featured chalk talk

Machine Learning at the Edge: Applications and Challenges
Machine learning at the TinyEdge is the way of the future but how we incorporate machine learning into our designs can take a variety of different forms. In this episode of Chalk Talk, Amelia chats with Dan Kozin from Silicon Labs about how you can add machine learning to your next design. They investigate what machine learning workflows look like, what machine learning tools you can utilize and the key challenges you will encounter as a machine learning developer.
Oct 20, 2022