feature article
Subscribe Now

Automotive Cameras Avoid the Pedestrian

In-car Sensors Deal with Problems You Didn’t Know You Had

“While the camera does not express the soul, perhaps a photograph can!” – Ansel Adams

Highway signs flicker. We just don’t notice it. 

This was one of the takeaways from a long discussion about camera sensors for automotive applications. Cars have cameras in them now, some outward facing and some inward facing. They’re used for different purposes (more on that in a moment), but the outward-facing ones have a slew of problems to deal with that I’d never considered. One of them is the nasty flickering of highway signs. 

Turns out, most electronic signs by major roads use LEDs, and contrary to all appearances, those LEDs aren’t really on all the time. They’re strobed at about 90 Hz, partly to save energy and manage brightness. That flickering is invisible to us; our eyes have enough persistence that we perceive the lights as solidly on. (Old-school fluorescent lights work much the same way.) 

This isn’t a problem for us humans, but it is if you’re a camera. Your average CMOS image sensor doesn’t have any visual persistence, so it will “see” the LEDs switch on and off several times per second. Depending on the frame rate and shutter speed of your camera, it might see the LEDs on all the time, off all the time, or some pseudo-random combination of the two. 

This is similar to the “helicopter effect,” where the rotational speed of the aircraft rotor happens to match the shutter speed of the film or video camera, making it look like the rotor is stationary even though the craft is flying. Your great-grandparents might have recognized this as a zoetrope

None of this amounts to more than a slight nuisance to videographers – unless you’re designing automotive safety systems. Then it’s a real problem. It isn’t just limited to exotic Level 5 self-driving systems, either. Even midrange safety systems that detect stop signs or speed limits can be fooled. What’s a poor designer to do? 

“Buy our image sensor,” says OmniVision, predictably enough. The company just announced its newest OX03C10 in-car camera chip that includes something called LFM: LED Flicker Mitigation. An image sensor’s exposure time is shorter than the illumination time of a roadside LED, so the ’C10 drags out exposure time to 11ms, takes multiple exposures, and does some internal magic that the company hesitates to describe. The result is that the image presented to downstream logic is flicker-free. Just like it looks to us.

OmniVision credits its stacked-die assembly for the improvement. Image sensors and digital logic want to be built on different semiconductor processes, so that’s what the company does. 

Headlights and illuminated billboards flicker, too, but these present a different problem. Recognition isn’t the issue; it’s displaying the video feed. If you pipe the incoming video to the car’s in-dash display, it will reproduce the flickering, which is ugly and distracting. It’s also weird, since we don’t perceive the original source that way. There must be something wrong with my car! So, systems have to filter out the flickering from all sources and present a smooth, but untrue, image of the world. The ’C10 takes care of that before the video leaves the chip. 

Looking the other way, some cars also have cameras pointed at the driver, usually to detect drowsy or distracted drivers. High-end cars have had this feature for the last ten years or so. The trick here is (a) to get a clear image of the driver, and (b) not to distract him/her in the process. Driver-monitoring cameras need to be small and discrete (preferably invisible), and they have to work in total darkness. Naturally, OmniVision has a new chip for this application, too. 

Since drivers don’t move around in their seat very fast, the DMS (driver monitoring system) camera can be pretty slow. Most use a rolling shutter, where each line of the image sensor is raster scanned. It takes a while to scan the whole sensor, but it’s plenty fast enough to capture a reasonable image. The trouble with slow scanning is that the light source (generally an infrared LED) needs to be illuminated the whole time, and that consumes more energy than the image sensor does. So, it makes sense from a power perspective to scan faster and turn the light off quickly.  

OmniVision’s new OVM9284 uses a global shutter, snapping the entire image in a few hundred microseconds. Each pixel is backed up with short-term storage (really just a capacitor) so that the image can be scanned out serially before it’s lost. 

OmniVision also replaced the traditional 850nm wavelength LED with a 940nm light source, on the theory that one is visible to the naked eye and one isn’t. It would be counterproductive to have a distracting red glow coming from your DMS camera while it’s monitoring you for distractions. 

Now, if we could just point those cameras the other way and make sure all the other drivers aren’t distracted. That would be a feature worth having. 

One thought on “Automotive Cameras Avoid the Pedestrian”

Leave a Reply

featured blogs
May 8, 2024
Learn how artificial intelligence of things (AIoT) applications at the edge rely on TSMC's N12e manufacturing processes and specialized semiconductor IP.The post How Synopsys IP and TSMC’s N12e Process are Driving AIoT appeared first on Chip Design....
May 2, 2024
I'm envisioning what one of these pieces would look like on the wall of my office. It would look awesome!...

featured video

Why Wiwynn Energy-Optimized Data Center IT Solutions Use Cadence Optimality Explorer

Sponsored by Cadence Design Systems

In the AI era, as the signal-data rate increases, the signal integrity challenges in server designs also increase. Wiwynn provides hyperscale data centers with innovative cloud IT infrastructure, bringing the best total cost of ownership (TCO), energy, and energy-itemized IT solutions from the cloud to the edge.

Learn more about how Wiwynn is developing a new methodology for PCB designs with Cadence’s Optimality Intelligent System Explorer and Clarity 3D Solver.

featured paper

Achieve Greater Design Flexibility and Reduce Costs with Chiplets

Sponsored by Keysight

Chiplets are a new way to build a system-on-chips (SoCs) to improve yields and reduce costs. It partitions the chip into discrete elements and connects them with a standardized interface, enabling designers to meet performance, efficiency, power, size, and cost challenges in the 5 / 6G, artificial intelligence (AI), and virtual reality (VR) era. This white paper will discuss the shift to chiplet adoption and Keysight EDA's implementation of the communication standard (UCIe) into the Keysight Advanced Design System (ADS).

Dive into the technical details – download now.

featured chalk talk

High Voltage Stackable Dual Phase Constant On Time Controllers - Microchip and Mouser
Sponsored by Mouser Electronics and Microchip
In this episode of Chalk Talk, Chris Romano from Microchip and Amelia Dalton discuss the what, where, and how of Microchip’s high voltage stackable dual phase constant on time controllers. They investigate the stacking capabilities of the MIC2132 controller, how these controllers compare with other solutions on the market, and how you can take advantage of these solutions in your next design.
May 22, 2023
40,800 views