feature article
Subscribe Now

Automotive Cameras Avoid the Pedestrian

In-car Sensors Deal with Problems You Didn’t Know You Had

“While the camera does not express the soul, perhaps a photograph can!” – Ansel Adams

Highway signs flicker. We just don’t notice it. 

This was one of the takeaways from a long discussion about camera sensors for automotive applications. Cars have cameras in them now, some outward facing and some inward facing. They’re used for different purposes (more on that in a moment), but the outward-facing ones have a slew of problems to deal with that I’d never considered. One of them is the nasty flickering of highway signs. 

Turns out, most electronic signs by major roads use LEDs, and contrary to all appearances, those LEDs aren’t really on all the time. They’re strobed at about 90 Hz, partly to save energy and manage brightness. That flickering is invisible to us; our eyes have enough persistence that we perceive the lights as solidly on. (Old-school fluorescent lights work much the same way.) 

This isn’t a problem for us humans, but it is if you’re a camera. Your average CMOS image sensor doesn’t have any visual persistence, so it will “see” the LEDs switch on and off several times per second. Depending on the frame rate and shutter speed of your camera, it might see the LEDs on all the time, off all the time, or some pseudo-random combination of the two. 

This is similar to the “helicopter effect,” where the rotational speed of the aircraft rotor happens to match the shutter speed of the film or video camera, making it look like the rotor is stationary even though the craft is flying. Your great-grandparents might have recognized this as a zoetrope

None of this amounts to more than a slight nuisance to videographers – unless you’re designing automotive safety systems. Then it’s a real problem. It isn’t just limited to exotic Level 5 self-driving systems, either. Even midrange safety systems that detect stop signs or speed limits can be fooled. What’s a poor designer to do? 

“Buy our image sensor,” says OmniVision, predictably enough. The company just announced its newest OX03C10 in-car camera chip that includes something called LFM: LED Flicker Mitigation. An image sensor’s exposure time is shorter than the illumination time of a roadside LED, so the ’C10 drags out exposure time to 11ms, takes multiple exposures, and does some internal magic that the company hesitates to describe. The result is that the image presented to downstream logic is flicker-free. Just like it looks to us.

OmniVision credits its stacked-die assembly for the improvement. Image sensors and digital logic want to be built on different semiconductor processes, so that’s what the company does. 

Headlights and illuminated billboards flicker, too, but these present a different problem. Recognition isn’t the issue; it’s displaying the video feed. If you pipe the incoming video to the car’s in-dash display, it will reproduce the flickering, which is ugly and distracting. It’s also weird, since we don’t perceive the original source that way. There must be something wrong with my car! So, systems have to filter out the flickering from all sources and present a smooth, but untrue, image of the world. The ’C10 takes care of that before the video leaves the chip. 

Looking the other way, some cars also have cameras pointed at the driver, usually to detect drowsy or distracted drivers. High-end cars have had this feature for the last ten years or so. The trick here is (a) to get a clear image of the driver, and (b) not to distract him/her in the process. Driver-monitoring cameras need to be small and discrete (preferably invisible), and they have to work in total darkness. Naturally, OmniVision has a new chip for this application, too. 

Since drivers don’t move around in their seat very fast, the DMS (driver monitoring system) camera can be pretty slow. Most use a rolling shutter, where each line of the image sensor is raster scanned. It takes a while to scan the whole sensor, but it’s plenty fast enough to capture a reasonable image. The trouble with slow scanning is that the light source (generally an infrared LED) needs to be illuminated the whole time, and that consumes more energy than the image sensor does. So, it makes sense from a power perspective to scan faster and turn the light off quickly.  

OmniVision’s new OVM9284 uses a global shutter, snapping the entire image in a few hundred microseconds. Each pixel is backed up with short-term storage (really just a capacitor) so that the image can be scanned out serially before it’s lost. 

OmniVision also replaced the traditional 850nm wavelength LED with a 940nm light source, on the theory that one is visible to the naked eye and one isn’t. It would be counterproductive to have a distracting red glow coming from your DMS camera while it’s monitoring you for distractions. 

Now, if we could just point those cameras the other way and make sure all the other drivers aren’t distracted. That would be a feature worth having. 

One thought on “Automotive Cameras Avoid the Pedestrian”

Leave a Reply

featured blogs
Dec 3, 2021
Hard to believe it's already December and 11/12ths of a year's worth of CFD is behind us. And with the holidays looming, it's uncertain how many more editions of This Week in CFD are... [[ Click on the title to access the full blog on the Cadence Community sit...
Dec 3, 2021
Explore automotive cybersecurity standards, news, and best practices through blog posts from our experts on connected vehicles, automotive SoCs, and more. The post How Do You Stay Ahead of Hackers and Build State-of-the-Art Automotive Cybersecurity? appeared first on From Si...
Dec 3, 2021
Believe it or not, I ran into John (he told me I could call him that) at a small café just a couple of evenings ago as I pen these words....
Nov 8, 2021
Intel® FPGA Technology Day (IFTD) is a free four-day event that will be hosted virtually across the globe in North America, China, Japan, EMEA, and Asia Pacific from December 6-9, 2021. The theme of IFTD 2021 is 'Accelerating a Smart and Connected World.' This virtual event ...

featured video

Synopsys & Samtec Demo PCIe 6.0 IP, Connector & Cable Systems for AI Hardware Designs

Sponsored by Synopsys

This demo features Synopsys’ DesignWare PHY IP for PCIe 6.0, performing at maximum channel loss, with Samtec's connectors in a configurable, GPU-based AI/ML system.

Click here for more information about DesignWare IP for PCI Express (PCIe) 6.0

featured paper

IPU-Based Cloud Infrastructure: The Fulcrum for Digital Business

Sponsored by Intel

As Cloud Service Providers consider their investment strategies and technology plans for the future, learn how IPUs can offer a path to accelerate and financially optimize cloud services.

Click to read more

featured chalk talk

KISSLING Products: Rugged and Reliable Solutions

Sponsored by Mouser Electronics and TE Connectivity

Rugged and reliable designs today have a specific set of design requirements that may not be found in other industries including robustness, durability, and the ability to resist harsh environments. In this episode of Chalk Talk, Amelia Dalton chats with Mark Dickson from TE Connectivity about the KISSLING product family which includes a wide variety of rugged and reliable solutions for your next design.

Click here for more information about TE Connectivity / KISSLING Ruggedized Switching Products