editor's blog
Subscribe Now

A New Coverage Concept

OneSpin announced a Quantify MDV product a few years back. With it, they defined a number of different coverage aspects – things that could be verified with their formal technology. Now they’ve reinforced that product with a new version. And that version contains yet another coverage concept.

The older coverage concepts focused on the design itself and the quality of stimulus used in verification. It would check for things like dead code and over-constraining, the former reflecting a possible code issue and the latter indicating that legitimate cases may not be covered by existing tests. I discussed these elements in my original coverage of the tool.

In recent times, they struggled a bit with what to call these checks. You might think they’re simply “design” checks, except for the constraining bits. The aspect that gets to simulation coverage had them calling it “simulation” coverage, but that didn’t really cut it either. They landed on “reachability,” since things like dead or redundant code indicated design elements that may or may not be reachable, and the constraints also get to whether or not certain failures can be reached by the tests. It’s not a perfect nomenclature, but, absent something perfect, it’s what they settled on.

Why even worry? Well, they needed to distinguish all of those coverage aspects from a new one they were adding. This new one tests the completeness of the assertions and checkers in the design. The assertions are designed to catch problems during formal verification, but it’s possible to write ineffective assertions. Looked at another way, if assertions are poor or incomplete, then there may be code failures that could never be observed by the assertions.

So they refer to this as “observation coverage.” And they test it using a form of “mutation” analysis: making a code change and seeing if the assertion picks it up. If not, then there may be a hole in the assertion.

This appears to be a newish concept, and it’s not comprehended in the UCIS coverage standard; they’re in discussions on that.

You can get a more complete picture of their latest Quantify release in their announcement.

Leave a Reply

featured blogs
May 25, 2018
You are probably subscribed to a number of email newsletters. No doubt you have been receiving emails saying that the system is changing the way that they are handling their mailing lists and that if you want to continue to receive the emails, then you need to re-subscribe. I...
May 24, 2018
Amazon has apparently had an Echo hiccup of the sort that would give customers bad dreams. It sent a random conversation to a random contact. A couple had installed numerous Alexa-enabled devices in the home. At some point, they had a conversation '€“ as couples are wont to...
May 23, 2018
It’s been over a year since we last updated the Samtec homepage. As you may have already noticed, we recently took some time to refresh it. You can see a comparison below of the old homepage (top), vs the new homepage (bottom). The new update puts more attention on our ...