feature article
Subscribe Now

A Benchmark That Measures Nothing

EEMBC and Volkswagen Add Sanity to Power-Saving Modes

Well, this is different: A benchmark that doesn’t measure performance. The newest benchmark to come out of EEMBC (the Embedded Microprocessor Benchmark Consortium) isn’t designed to measure MIPS or megahertz, but instead is geared toward quantifying a chip’s power-saving modes. And Volkswagen and other automakers are deeply interested in the outcome.

Here’s what happened. VW, like all automakers, consumes a bunch of microcontrollers in its cars. In addition to the familiar VW brand, VW also owns Audi, Lamborghini, Bentley, Porsche, Bugatti, the Spanish brand SEAT, the Eastern European Skoda nameplate, Scania trucks, and Ducati motorcycles. In short, the company makes a lot of vehicles, from very high-end exotics to very low-end commuter boxes.

And you know what problem VW’s engineers face? They can’t make sense of microcontrollers’ low-power modes.

I can see their point. Every MCU seems to have two or three different power-saving modes, from “sleep” and “snooze” to “hibernate” or “comatose.” No two sleep modes are the same, and there’s no standardization whatsoever on what they’re called or how they work. An engineer can easily spend hours or days scrutinizing the datasheet to figure out what power-saving modes will work best for them – and that’s assuming all the claims made in the datasheet are correct and relevant to your application. Factor in another few weeks for testing the various modes, and you might develop enough confidence to actually use the chip in your system.

Then, of course, all that research goes out the window when you upgrade to the next chip.

Back to VW. The company scatters microcontrollers all over its vehicles, from engine and powertrain controllers to MCUs that handle the rear view mirror (seriously). The average car has over a dozen MCUs in it, and a high-end Audi, Lamborghini, or Bugatti can easily top 100. Most of those MCUs sleep most of the time. The chip that monitors the trunk to see if it’s latched, for example, might ping the sensor only once per second. The rest of the time, it’s monitoring the car’s battery temperature, or checking that the taillights aren’t burnt out, or – more than likely – just asleep. With dozens of similar MCUs dotted all over the car, their power consumption starts to add up. This is especially troubling with the increased emphasis on gas mileage and fuel efficiency, and doubly troubling when you get to electric vehicles (which VW is also developing). It’s not hard to eat up a couple of horsepower just generating DC current for all those microcontrollers.

Performance isn’t a big deal for most of these MCUs. After all, how hard is it to check the trunk latch? But sleep-mode power consumption is, and so is wakeup time. Does the chip retain all of its internal state information in sleep mode, or just the SRAM contents? Can I wake it with a timer, or does it require an interrupt? Will it awaken fast enough to catch traffic on the CAN bus, or do I need to keep it awake more often? All these questions, and more, were troubling VW’s engineers.

In a reversal of roles, VW approached EEMBC to ask for help developing a suite of benchmarks that would help the German automaker make sense of rival MCU vendors’ competing claims. VW supplied some of its own sensor data gleaned from in-house testing, while EEMBC came up with a vendor-neutral way to take that input and provide some sort of score for grading power modes.

Here’s how the test works: An MCU under scrutiny will have its inputs (say, CAN or an ADC) connected to a signal generator that mimics VW’s real-world data. The MCU vendor (Microchip, Renesas, Freescale, et al) gets to decide which of its many available power-saving modes is most suitable for that application. EEMBC randomizes the input data a little bit to prevent gaming the system, and checks to see if the MCU wakes up in time to perform all the necessary tasks, and, if so, how much energy it consumed. Assuming the chip passes, VW might (or might not) award the win to the most power-efficient chip. If the MCU fails for any reason, it’s hasta la vista, baby and on to the next MCU.

Why wouldn’t VW always pick the most power-efficient chip? For the same reasons you don’t. Sometimes it’s better to stick with one particular chip family or architecture because you prefer their development tools, or you get a break on pricing, or you just like the salesperson. Benchmarks don’t dictate behavior, they just inform our decisions. And VW has to make a lot of these decisions.

Remarkably, the company often clings to older MCUs and avoids upgrading simply because of the drama involved in qualifying a new chip. New silicon inevitably has newer, better power-saving modes, but it’s such a nuisance to document and test them that VW often doesn’t bother. They’re leaving energy efficiency on the table because it’s too much trouble to quantify it. That’s the real value of EEMBC’s new benchmark suite. It makes new chips interesting again, instead of an irritation. And that should, uh, drive more innovation. 

Leave a Reply

featured blogs
Oct 19, 2021
Learn about key roadblocks to improve ADAS systems & higher levels of autonomous driving, such as SoC performance, from our 2021 ARC Processor Virtual Summit. The post Top 5 Challenges to Achieve High-Level Automated Driving appeared first on From Silicon To Software....
Oct 19, 2021
Today, at CadenceLIVE Europe, we announced the Cadence Safety Solution, a new offering targeting safety-critical applications and featuring integrated analog and digital safety flows and engines for... [[ Click on the title to access the full blog on the Cadence Community si...
Oct 13, 2021
How many times do you search the internet each day to track down for a nugget of knowhow or tidbit of trivia? Can you imagine a future without access to knowledge?...
Oct 4, 2021
The latest version of Intel® Quartus® Prime software version 21.3 has been released. It introduces many new intuitive features and improvements that make it easier to design with Intel® FPGAs, including the new Intel® Agilex'„¢ FPGAs. These new features and improvements...

featured video

Intel Architecture Day 2021: Data Center - Infrastructure Processing Unit

Sponsored by Intel

Intel unveiled its biggest architectural shifts in a generation for CPUs, GPUs and IPUs to satisfy the crushing demand for more compute performance at Architecture Day 2021. Guido Appenzeller, Chief Technology Officer of Intel's Data Platforms Group explains how the IPU's design enables cloud and communication service providers to reduce overhead and free up performance for central processing units.

Click here to learn more

featured paper

Ultra Portable IO On The Go

Sponsored by Maxim Integrated (now part of Analog Devices)

The Go-IO programmable logic controller (PLC) reference design (MAXREFDES212) consists of multiple software configurable IOs in a compact form factor (less than 1 cubic inch) to address the needs of industrial automation, building automation, and industrial robotics. Go-IO provides design engineers with the means to rapidly create and prototype new industrial control systems before they are sourced and constructed.

Click to read more

featured chalk talk

Accelerating Innovation at the Edge with Xilinx Adaptive System on Modules

Sponsored by Xilinx

The combination of system-on-module technology with advanced SoCs with programmable logic offer the ultimate in functionality, performance, flexibility, power efficiency, and ease of use. In this episode of Chalk Talk, Amelia Dalton chats with Karan Kantharia of Xilinx about the new Kira SOM, and how it enables faster time-to-deployment versus conventional component-based design.

Click here for more information about Kria Adaptive System-on-Modules