feature article
Subscribe Now

Storm the Barricades and Pro Test

New EEMBC Pro Benchmark Tests Higher Level Processors

Those darn processors. They keep getting faster and we keep running out of ways to keep track of them.

In the beginning, there was clock speed. Faster clock meant faster processor, right? Not so fast. Some processors – like some people – did more work than others in a given amount of time. So your 5-KHz UNIVAC might or might not be faster than my 3-KHz ENIAC.

So we created benchmarks. Let’s run LINPAC or Whetstone on both of ’em and see who finishes first. That was better, but it still told you only how well your computer ran that test. Not how well it would run any test. Maybe your machine is designed for integer arithmetic while mine is geared toward floating-point math.

So the benchmarks got more complicated, clever, and creative. And componentized. Instead of just one test, we got test suites: bundles of benchmarks that ostensibly tested everything before divulging the details. Much better.

But, but… But your benchmark doesn’t measure memory latency. And my benchmark doesn’t exercise graphics performance. And the industry-standard benchmarks don’t record power consumption, HTML rendering time, or Java performance (an oxymoron).

And the vendors never quote any of those benchmarks in their literature, anyway. They’re still publishing Dhrystone MIPS ratings, as if we’re still in the 1970s. That’s like scoring BMX half-pipe with stone tablets or timing the 400-meter dash with a sundial.

Since the time of EEMBC (the Embedded Microprocessor Benchmark Consortium – the first E is silent – or maybe the second), things have gotten a lot better. The nonprofit group has done a bang-up job of developing all sorts of free benchmarks for all different kinds of technology, market niches, and applications. Chief among these was CoreMark – the main, or core, benchmark in most of its CPU measurements. And the microprocessor world looked upon CoreMark and saw that it was good.

Good, but not great. Trouble is, CoreMark is a fixed benchmark (duh), but the chips it’s testing get ever more complex. So it was just a matter of time before CoreMark was outmatched by the chips it was intended to quantify. For instance, CoreMark doesn’t really do multicore processors. Oh, sure, it’ll run on multicore chips. But it doesn’t really exercise the multi-core-ness. It’s not multithreaded, and it wasn’t really intended to measure parallel processing.

It is also – how shall we say it – small. Like Dhrystone before it (way before it), CoreMark now fits inside the cache of some larger processors. That eliminates the effects of memory latency and bandwidth, which partially defeats the purpose. Sometimes even L2 or L3 caches went untouched. So CoreMark needed to beef up and bulk up in order to meet the demands of today’s demanding processors.

Voila! Right on cue, CoreMark 2.0 arrives. Or, as EEMBC officially calls it, CoreMark-Pro. Despite the name, the Pro version is still free and still available to anyone who wants to download and run it. But it’s much more intensive than before.

Where the original CoreMark had a single integer workload, CoreMark-Pro has five, plus four floating-point workloads, for a total of nine. Each of the nine individual tests is bigger than the original CoreMark, so you’re expected to run them all separately, not all at once. That yields nine technology-specific scores, which Pro will obligingly combine into a single CoreMark-Pro score. (Everyone likes simple, single scores.)

Does it run on multicore chips? Yes. How? That’s up to you. EEMBC believes that task distribution is the proper responsibility of the compiler, developer, operating system, or magical tool – whatever you prefer. Hard-coding CoreMark-Pro for vectorization would have unintentionally favored chips that implemented the same sort of parallelism. Instead, the code is generic, in the sense that it’s easily compiled just like anybody else’s code. However you like to distribute code threads in real life is how you should do it with CoreMark-Pro. Just as with compiler optimizations, thread distribution is likely to affect performance quite a bit. Expect a lot of tweaking in this area, but that’s okay. What’s convenient for EEMBC is also fairer for everyone else. More scalable, too. 

CoreMark-Pro does not, in fact, owe much to the original CoreMark. None of the code was reused from its older sibling, although one of the five integer tests is pretty similar to CoreMark in the type of work that it does. As before, users are free to download the source code and free to post their results anywhere they wish. No certification or approval from EEMBC is necessary; cheating is handled strictly on the honor system. If nobody else can verify your CoreMark-Pro scores, the bad Yelp reviews will shame you into compliance.

However, if you desire the imprimatur of third-party certification, EEMBC can perform the task for a nominal fee. But you’ll have to take a number; the waiting list is long and the testing lab has finite resources.

And what becomes of the old CoreMark? It lives on, happily providing a baseline for testing smaller processors (by today’s standards). Changing it now would instantly throw the 500+ published scores out the window. Nobody wants that, so CoreMark 1.0 is now the tool for low-end and midrange chips, while CoreMark-Pro tackles the big iron. For now. It can’t be long before CoreMark 3.0 is in the works

Leave a Reply

featured blogs
Jul 25, 2021
https://youtu.be/cwT7KL4iShY Made on "a tropical beach" Monday: Aerospace and Defense Systems Day...and DAU Tuesday: 75 Years of the Microprocessor Wednesday: CadenceLIVE Cloud Panel... [[ Click on the title to access the full blog on the Cadence Community site. ]]...
Jul 24, 2021
Many modern humans have 2% Neanderthal DNA in our genomes. The combination of these DNA snippets is like having the ghost of a Neanderthal in our midst....
Jul 23, 2021
Synopsys co-CEO Aart de Geus explains how AI has become an important chip design tool as semiconductor companies continue to innovate in the SysMoore Era. The post Entering the SysMoore Era: Synopsys Co-CEO Aart de Geus on the Need for AI-Designed Chips appeared first on Fro...
Jul 9, 2021
Do you have questions about using the Linux OS with FPGAs? Intel is holding another 'Ask an Expert' session and the topic is 'Using Linux with Intel® SoC FPGAs.' Come and ask our experts about the various Linux OS options available to use with the integrated Arm Cortex proc...

featured video

Adopt a Shift-left Methodology to Accelerate Your Product Development Process

Sponsored by Cadence Design Systems

Validate your most sophisticated SoC designs before silicon and stay on schedule. Balance your workload between simulation, emulation and prototyping for complete system validation. You need the right tool for the right job. Emulation meets prototyping -- Cadence Palladium and Protium Dynamic Duo for IP/SoC verification, hardware and software regressions, and early software development.

More information about Emulation and Prototyping

featured paper

Hyperconnectivity and You: A Roadmap for the Consumer Experience

Sponsored by Cadence Design Systems

Will people’s views about hyperconnectivity and hyperscale computing affect requirements for your next system or IC design? Download the latest Cadence report for how consumers view hyperscale computing’s impact on cars, mobile devices, and health.

Click to read more

featured chalk talk

ROHM's KX132-1211 & KX134-1211 Accelerometers

Sponsored by Mouser Electronics and ROHM Semiconductor

Machine health monitoring is a key benefit in the Industry 4.0 revolution. Integrating data from sensors for vibration detection, motion detection, angle measurement and more can give a remarkably accurate picture of machine health, and timely warning of impending failure. In this episode of Chalk Talk, Amelia Dalton chats with Alex Chernyakov of ROHM Semiconductor about the key considerations in machine health monitoring, and how a new line of accelerometers for industrial applications can help.

Click here for more information about Kionix / ROHM Semiconductor KX134 & KX132 Tri-axis Digital Accelerometers