feature article
Subscribe Now

Storm the Barricades and Pro Test

New EEMBC Pro Benchmark Tests Higher Level Processors

Those darn processors. They keep getting faster and we keep running out of ways to keep track of them.

In the beginning, there was clock speed. Faster clock meant faster processor, right? Not so fast. Some processors – like some people – did more work than others in a given amount of time. So your 5-KHz UNIVAC might or might not be faster than my 3-KHz ENIAC.

So we created benchmarks. Let’s run LINPAC or Whetstone on both of ’em and see who finishes first. That was better, but it still told you only how well your computer ran that test. Not how well it would run any test. Maybe your machine is designed for integer arithmetic while mine is geared toward floating-point math.

So the benchmarks got more complicated, clever, and creative. And componentized. Instead of just one test, we got test suites: bundles of benchmarks that ostensibly tested everything before divulging the details. Much better.

But, but… But your benchmark doesn’t measure memory latency. And my benchmark doesn’t exercise graphics performance. And the industry-standard benchmarks don’t record power consumption, HTML rendering time, or Java performance (an oxymoron).

And the vendors never quote any of those benchmarks in their literature, anyway. They’re still publishing Dhrystone MIPS ratings, as if we’re still in the 1970s. That’s like scoring BMX half-pipe with stone tablets or timing the 400-meter dash with a sundial.

Since the time of EEMBC (the Embedded Microprocessor Benchmark Consortium – the first E is silent – or maybe the second), things have gotten a lot better. The nonprofit group has done a bang-up job of developing all sorts of free benchmarks for all different kinds of technology, market niches, and applications. Chief among these was CoreMark – the main, or core, benchmark in most of its CPU measurements. And the microprocessor world looked upon CoreMark and saw that it was good.

Good, but not great. Trouble is, CoreMark is a fixed benchmark (duh), but the chips it’s testing get ever more complex. So it was just a matter of time before CoreMark was outmatched by the chips it was intended to quantify. For instance, CoreMark doesn’t really do multicore processors. Oh, sure, it’ll run on multicore chips. But it doesn’t really exercise the multi-core-ness. It’s not multithreaded, and it wasn’t really intended to measure parallel processing.

It is also – how shall we say it – small. Like Dhrystone before it (way before it), CoreMark now fits inside the cache of some larger processors. That eliminates the effects of memory latency and bandwidth, which partially defeats the purpose. Sometimes even L2 or L3 caches went untouched. So CoreMark needed to beef up and bulk up in order to meet the demands of today’s demanding processors.

Voila! Right on cue, CoreMark 2.0 arrives. Or, as EEMBC officially calls it, CoreMark-Pro. Despite the name, the Pro version is still free and still available to anyone who wants to download and run it. But it’s much more intensive than before.

Where the original CoreMark had a single integer workload, CoreMark-Pro has five, plus four floating-point workloads, for a total of nine. Each of the nine individual tests is bigger than the original CoreMark, so you’re expected to run them all separately, not all at once. That yields nine technology-specific scores, which Pro will obligingly combine into a single CoreMark-Pro score. (Everyone likes simple, single scores.)

Does it run on multicore chips? Yes. How? That’s up to you. EEMBC believes that task distribution is the proper responsibility of the compiler, developer, operating system, or magical tool – whatever you prefer. Hard-coding CoreMark-Pro for vectorization would have unintentionally favored chips that implemented the same sort of parallelism. Instead, the code is generic, in the sense that it’s easily compiled just like anybody else’s code. However you like to distribute code threads in real life is how you should do it with CoreMark-Pro. Just as with compiler optimizations, thread distribution is likely to affect performance quite a bit. Expect a lot of tweaking in this area, but that’s okay. What’s convenient for EEMBC is also fairer for everyone else. More scalable, too. 

CoreMark-Pro does not, in fact, owe much to the original CoreMark. None of the code was reused from its older sibling, although one of the five integer tests is pretty similar to CoreMark in the type of work that it does. As before, users are free to download the source code and free to post their results anywhere they wish. No certification or approval from EEMBC is necessary; cheating is handled strictly on the honor system. If nobody else can verify your CoreMark-Pro scores, the bad Yelp reviews will shame you into compliance.

However, if you desire the imprimatur of third-party certification, EEMBC can perform the task for a nominal fee. But you’ll have to take a number; the waiting list is long and the testing lab has finite resources.

And what becomes of the old CoreMark? It lives on, happily providing a baseline for testing smaller processors (by today’s standards). Changing it now would instantly throw the 500+ published scores out the window. Nobody wants that, so CoreMark 1.0 is now the tool for low-end and midrange chips, while CoreMark-Pro tackles the big iron. For now. It can’t be long before CoreMark 3.0 is in the works

Leave a Reply

featured blogs
Jul 6, 2020
If you were in the possession of one of these bodacious beauties, what sorts of games and effects would you create using the little scamp?...
Jul 3, 2020
[From the last episode: We looked at CNNs for vision as well as other neural networks for other applications.] We'€™re going to take a quick detour into math today. For those of you that have done advanced math, this may be a review, or it might even seem to be talking down...
Jul 2, 2020
In June, we continued to upgrade several key pieces of content across the website, including more interactive product explorers on several pages and a homepage refresh. We also made a significant update to our product pages which allows logged-in users to see customer-specifi...

Featured Video

Product Update: Advances in DesignWare Die-to-Die PHY IP

Sponsored by Synopsys

Hear the latest about Synopsys' DesignWare Die-to-Die PHY IP for SerDes-based 112G USR/XSR and parallel-based HBI interfaces. The IP, available in advanced FinFET processes, addresses the power, bandwidth, and latency requirements of high-performance computing SoCs targeting hyperscale data center, AI, and networking applications.

Click here for more information about DesignWare Die-to-Die PHY IP Solutions

Featured Paper

Cryptography: A Closer Look at the Algorithms

Sponsored by Maxim Integrated

Get more details about how cryptographic algorithms are implemented and how an asymmetric key algorithm can be used to exchange a shared private key.

Click here to download the whitepaper

Featured Chalk Talk

Automotive MOSFET for the Transportation Market

Sponsored by Mouser Electronics and Infineon

MOSFETS are critical in automotive applications, where long-term reliability is paramount. But, do we really understand the failure rates and mechanisms in the devices we design in? In this episode of Chalk Talk, Amelia Dalton sits down with Jeff Darrow of Infineon to discuss the role of MOSFETS in transportation, solder inspection, qualification.

Click here for more information about Infineon Technologies OptiMOS™ 5 Power MOSFETs