OneSpin announced a Quantify MDV product a few years back. With it, they defined a number of different coverage aspects – things that could be verified with their formal technology. Now they’ve reinforced that product with a new version. And that version contains yet another coverage concept.
The older coverage concepts focused on the design itself and the quality of stimulus used in verification. It would check for things like dead code and over-constraining, the former reflecting a possible code issue and the latter indicating that legitimate cases may not be covered by existing tests. I discussed these elements in my original coverage of the tool.
In recent times, they struggled a bit with what to call these checks. You might think they’re simply “design” checks, except for the constraining bits. The aspect that gets to simulation coverage had them calling it “simulation” coverage, but that didn’t really cut it either. They landed on “reachability,” since things like dead or redundant code indicated design elements that may or may not be reachable, and the constraints also get to whether or not certain failures can be reached by the tests. It’s not a perfect nomenclature, but, absent something perfect, it’s what they settled on.
Why even worry? Well, they needed to distinguish all of those coverage aspects from a new one they were adding. This new one tests the completeness of the assertions and checkers in the design. The assertions are designed to catch problems during formal verification, but it’s possible to write ineffective assertions. Looked at another way, if assertions are poor or incomplete, then there may be code failures that could never be observed by the assertions.
So they refer to this as “observation coverage.” And they test it using a form of “mutation” analysis: making a code change and seeing if the assertion picks it up. If not, then there may be a hole in the assertion.
This appears to be a newish concept, and it’s not comprehended in the UCIS coverage standard; they’re in discussions on that.
You can get a more complete picture of their latest Quantify release in their announcement.
