feature article
Subscribe Now

Automotive Cameras Avoid the Pedestrian

In-car Sensors Deal with Problems You Didn’t Know You Had

“While the camera does not express the soul, perhaps a photograph can!” – Ansel Adams

Highway signs flicker. We just don’t notice it. 

This was one of the takeaways from a long discussion about camera sensors for automotive applications. Cars have cameras in them now, some outward facing and some inward facing. They’re used for different purposes (more on that in a moment), but the outward-facing ones have a slew of problems to deal with that I’d never considered. One of them is the nasty flickering of highway signs. 

Turns out, most electronic signs by major roads use LEDs, and contrary to all appearances, those LEDs aren’t really on all the time. They’re strobed at about 90 Hz, partly to save energy and manage brightness. That flickering is invisible to us; our eyes have enough persistence that we perceive the lights as solidly on. (Old-school fluorescent lights work much the same way.) 

This isn’t a problem for us humans, but it is if you’re a camera. Your average CMOS image sensor doesn’t have any visual persistence, so it will “see” the LEDs switch on and off several times per second. Depending on the frame rate and shutter speed of your camera, it might see the LEDs on all the time, off all the time, or some pseudo-random combination of the two. 

This is similar to the “helicopter effect,” where the rotational speed of the aircraft rotor happens to match the shutter speed of the film or video camera, making it look like the rotor is stationary even though the craft is flying. Your great-grandparents might have recognized this as a zoetrope

None of this amounts to more than a slight nuisance to videographers – unless you’re designing automotive safety systems. Then it’s a real problem. It isn’t just limited to exotic Level 5 self-driving systems, either. Even midrange safety systems that detect stop signs or speed limits can be fooled. What’s a poor designer to do? 

“Buy our image sensor,” says OmniVision, predictably enough. The company just announced its newest OX03C10 in-car camera chip that includes something called LFM: LED Flicker Mitigation. An image sensor’s exposure time is shorter than the illumination time of a roadside LED, so the ’C10 drags out exposure time to 11ms, takes multiple exposures, and does some internal magic that the company hesitates to describe. The result is that the image presented to downstream logic is flicker-free. Just like it looks to us.

OmniVision credits its stacked-die assembly for the improvement. Image sensors and digital logic want to be built on different semiconductor processes, so that’s what the company does. 

Headlights and illuminated billboards flicker, too, but these present a different problem. Recognition isn’t the issue; it’s displaying the video feed. If you pipe the incoming video to the car’s in-dash display, it will reproduce the flickering, which is ugly and distracting. It’s also weird, since we don’t perceive the original source that way. There must be something wrong with my car! So, systems have to filter out the flickering from all sources and present a smooth, but untrue, image of the world. The ’C10 takes care of that before the video leaves the chip. 

Looking the other way, some cars also have cameras pointed at the driver, usually to detect drowsy or distracted drivers. High-end cars have had this feature for the last ten years or so. The trick here is (a) to get a clear image of the driver, and (b) not to distract him/her in the process. Driver-monitoring cameras need to be small and discrete (preferably invisible), and they have to work in total darkness. Naturally, OmniVision has a new chip for this application, too. 

Since drivers don’t move around in their seat very fast, the DMS (driver monitoring system) camera can be pretty slow. Most use a rolling shutter, where each line of the image sensor is raster scanned. It takes a while to scan the whole sensor, but it’s plenty fast enough to capture a reasonable image. The trouble with slow scanning is that the light source (generally an infrared LED) needs to be illuminated the whole time, and that consumes more energy than the image sensor does. So, it makes sense from a power perspective to scan faster and turn the light off quickly.  

OmniVision’s new OVM9284 uses a global shutter, snapping the entire image in a few hundred microseconds. Each pixel is backed up with short-term storage (really just a capacitor) so that the image can be scanned out serially before it’s lost. 

OmniVision also replaced the traditional 850nm wavelength LED with a 940nm light source, on the theory that one is visible to the naked eye and one isn’t. It would be counterproductive to have a distracting red glow coming from your DMS camera while it’s monitoring you for distractions. 

Now, if we could just point those cameras the other way and make sure all the other drivers aren’t distracted. That would be a feature worth having. 

One thought on “Automotive Cameras Avoid the Pedestrian”

Leave a Reply

featured blogs
Jul 3, 2020
[From the last episode: We looked at CNNs for vision as well as other neural networks for other applications.] We'€™re going to take a quick detour into math today. For those of you that have done advanced math, this may be a review, or it might even seem to be talking down...
Jul 2, 2020
Using the bitwise operators in general -- and employing them to perform masking, bit testing, and bit setting/clearing operations in particular -- can be extremely efficacious....
Jul 2, 2020
In June, we continued to upgrade several key pieces of content across the website, including more interactive product explorers on several pages and a homepage refresh. We also made a significant update to our product pages which allows logged-in users to see customer-specifi...

Featured Video

Product Update: DesignWare® TCAM IP -- Synopsys

Sponsored by Synopsys

Join Rahul Thukral in this discussion on TCAMs, including performance and power considerations. Synopsys TCAMs are used in networking and automotive applications as they are low-risk, production-proven, and meet automotive requirements.

Click here for more information about DesignWare Foundation IP: Embedded Memories, Logic Libraries & GPIO

Featured Paper

Cryptography: Fundamentals on the Modern Approach

Sponsored by Maxim Integrated

Learn about the fundamental concepts behind modern cryptography, including how symmetric and asymmetric keys work to achieve confidentiality, identification and authentication, integrity, and non-repudiation.

Click here to download the whitepaper

Featured Chalk Talk

Maxim's Himalaya uSLIC Portfolio

Sponsored by Mouser Electronics and Maxim Integrated

With form factors continuing to shrink, most engineers are working hard to reduce the number of discrete components in their designs. Power supplies, in particular, are problematic - often requiring a number of large components. In this episode of Chalk Talk, Amelia Dalton chats with John Woodward of Maxim Integrated about how power modules can save board space, improve performance, and help reliability.

Click here for more information about Maxim Integrated Himalaya uSLIC™ MAXM1546x Step-Down Power Modules