“If debugging is the act of removing bugs, then programming must be the act of inserting them.” So goes the old axiom about embedded development. It’s true that most programmers spend more time debugging software than they do writing the code in the first place – several surveys have proven this dreary division of labor. So if we spend so much time debugging, why aren’t we better at it?
Kozio aims to speed up that process. The Colorado-based company makes something it calls VTOS (verification and test operating system), a kind of overall wrapper for test-and-debug code. I’m not convinced that calling VTOS an “operating system” is a good move, though. To me, operating systems are large and complex, and VTOS isn’t that. On the other hand, VTOS is very helpful and definitely a step up from embedded debug monitors. So withhold judgment for a moment as we plumb the depths of VTOS and see what it can do for you.
First off, VTOS does several different things, and you’ll likely use it at different points in your development process. Putting last things first, you can burn VTOS into your finished product’s ROM and use it to remotely diagnose problems in the field. You can also use it to decide whether your customer is having a hardware problem or a software problem. Earlier on, you can use VTOS during your company’s production phase to decide whether two boards are identical. For example, it can determine whether a stack of boards has been updated to the latest hardware revision. It can also help you decide whether two components are really functionally identical or if there are some vendor-specific differences. Even earlier on, you can use VTOS to help bring up new hardware, which is especially handy when the software guys are waiting for prototypes.
How does VTOS do all this? With a little help from its friends, namely you. I think of VTOS as a kind of overseer, a manager of board-specific tests that you write yourself. In the revision-comparison example above, you’d need to write a little program that can tell the difference(s) between two board revisions. Once you’ve done that, VTOS will make sure that test gets downloaded and run on the boards in question and report back on the results. That’s a trivial example, but it starts to give you an idea of where VTOS fits in the overall debug world. It handles multiple hardware-specific tests, starts and runs them, downloads them over Ethernet, serial, or JTAG links, reports results, peeks and pokes into memory and I/O, runs scripts, and collects statistics. It’s a standalone debugger, it’s a bootstrap downloader, and it’s a test harness. I guess maybe the “operating system” description is accurate after all.
VTOS is also remarkably versatile and configurable. The trick with any early-stage hardware debugger is making it understand your hardware. How does it know what processor, UART, FPGA, daughter board, and other features are on it? Here’s where Kozio got clever. The company has ported VTOS to a number of popular embedded processors, such as Freescale’s PowerPC-based QorIQ processors, TI ARM-based processors, Intel chips, and so on. To that they add a big laundry list of popular peripheral chips. Before you run VTOS for the first time, you simply pick and choose your processor and peripherals off of the extensive menu, assign them all addresses, and let the VTOS auto-configuration take over. It produces a downloadable binary specifically for your board. No porting required; just choose options from the menu. Most customers get up and running in about 30 minutes, even on brand new, never-before-seen hardware. Pretty slick.
Want to run a series of debug, compatibility, or verification tests? VTOS can script multiple runtime images and handle starting, stopping, and reporting their progress. Scripts can be as simple as an ASCII file you create with Windows Notepad and drag and drop to the VTOS icon on your PC. Want to make certain tests or scripts permanent? VTOS can burn them into your ROM or flash image so they’re always available.
For the detail-oriented, VTOS has a command-line interface that allows you to peek and poke individual addresses, trace execution, whip out some quick code, or manually download new tests. For less skilled operators, premade scripts can manage a suite of tests and provide a simple pass/fail indication. In the field, tech support can remotely activate VTOS, access built-in tests, or initiate downloads for more elaborate one-off diagnostics.
Calling it an “operating system” suggests that VTOS takes the place of Linux, VxWorks, uCOS, Android, or whatever. While that’s probably true during the early debug phases, VTOS usually lives alongside a real operating system later on. Sometimes boards will dual-boot either VTOS or (for instance) Linux. Other times, VTOS is buried in ROM and accessible only through a special start-up sequence. Finally, VTOS can be left off the board entirely and downloaded only when needed, via something like Ethernet or a JTAG port.
To your end customer, VTOS will be completely invisible and irrelevant. But to you and your colleagues, it may be the most useful code on the board. Anything that helps to bring up new hardware faster, crank out production quicker, or diagnose remote problems more thoroughly has got to be a good thing. After all, that’s apparently where we spend most of our time. Might as well enjoy it.
