feature article
Subscribe Now

Undo Your Mistakes at the Customer’s Site

Debugging Software Now Works Remotely

“Programming will be the last job on the planet.” – Jaan Tallinn, co-creator of Skype and Kazaa

Even hardware is software these days. With a hardware-development language (HDL), you can “compile” gates, boards, and even entire systems. Not everybody does… but you could. Software determines product functionality at least as much as the hardware does, and programming is a much easier skill to learn (and to teach) than hardware engineering. We’re becoming an industry of coders.

Yet one of the big downsides to programming is that you generally spend more time debugging your code than you do creating it. That’s the consistent conclusion of repeated surveys of professional programmers from around the world: they spend more time debugging than on any other part of the job.

Or maybe that’s actually a good thing in disguise. Hardware engineers generally spend less time (as a percentage) on debugging than their coding colleagues do, but that’s likely because hardware bugs are so expensive and time-consuming to fix. You can’t just iteratively debug a new 10-million-gate SoC by fixing little things here and there every half hour. It’s got to be pretty much perfect the first time. Consequently, hardware folks spend a lot of time up front simulating and verifying their design before they push the big red button. Programmers? Not so much. We’ll fix it as we go along. Or maybe sometime after that. We’ll see.

The result is a lot of iffy code that gets shipped out to the customer, with a vague promise of a fix, an update, or an upgrade somewhere down the road. Why hold up revenue when we can patch bugs in the field? It’s the end of the quarter; let’s make our number and worry about the update later.

This all assumes that you already know where the bugs are, however. It’s all very nice to push planned updates to your customers. But what about real bugs – the unexpected kind? What happens when tech support starts getting calls and emails about some failure you’ve never heard of and never anticipated? Now you’ve got a real problem. You’ve got systems installed in the field that are failing in ways you can’t duplicate in the lab. It’s too late to call the faulty systems back. And all the units in-house seem to work just fine. What’s going on at the customers’ site that’s making these things fail?

Who you gonna call?

Undo Software, that’s who. Or at least, they’d like you to give them a chance. Undo is a very small (12-person) company in Cambridge, UK that has been peddling a debug tool for a few years. It’s not a standalone debugger itself, but an enhancement for several popular debuggers that allows you to “rewind” program execution, one instruction at a time. It’s essentially a time machine for code. You can stop execution, back it up, roll it forward again, and see every pointer, register, and machine instruction along the way. It’s an obsessive amount of detail, but sometimes that’s exactly what you want.

The tool works by logging every processor activity and memory reference and squirreling it away in a circular buffer that you set aside in your system memory. There is some processor overhead: “You can’t record everything a processor does, instruction by instruction, without incurring some overhead. But we’re clever about minimizing it,” says company cofounder Greg Law. The debug code itself occupies “a few megabytes” of code space. The size of the buffer is up to you, but a few more megabytes is a good starting point. The tool runs only on Linux or Android systems, either ARM- or x86-based.

That’s the old product. The new product, just announced this month, allows you to collapse both time and space. In other words, it lets you rewind code on a remote system – say at your customer’s site. Up until now, you could use Undo’s magic time machine only on a local system sitting in front of you. Now you can operate it remotely, too.

The remote capability gives developers a handy new tool for cracking open buggy systems that are out of reach. Simply include Undo’s special “Live Recorder” library in your shipping code, and leave it dormant until the bug day comes. Then, activate it remotely and voila! Spooky action at a distance.

The remote version logs all the same data, and gives you all the same detailed visibility, as the local version. It’s essentially a headless version of the earlier product, with the addition of up/downloading capability and some intrinsic privacy. Why privacy features on a debugger? Two reasons. One, you don’t want just any old programmer examining your detailed trace logs. Given the level of detail it records, UndoDB would be an ideal tool for reverse-engineering code. It would also be a handy cracking tool, were you so inclined.

The second reason has more to do with commerce. Undo licenses its software on an annual basis: buy one license and deploy as many copies as you like within a single project/product. No royalties; no need to count systems. That’s very convenient, but it also makes it hard for Undo to get paid. To thwart unscrupulous reuse of its code, Undo serializes each copy of Live Recorder, and the linked library will communicate only with its “twin” back at your desk. That allows you to talk to your remote system, but it keeps other people (and even other Undo Live Recorder users) out.

So now you can feel slightly less guilty about debugging code after it’s shipped to the customer. 

Leave a Reply

featured blogs
Nov 24, 2020
In our last Knowledge Booster Blog , we introduced you to some tips and tricks for the optimal use of the Virtuoso ADE Product Suite . W e are now happy to present you with some further news from our... [[ Click on the title to access the full blog on the Cadence Community s...
Nov 23, 2020
It'€™s been a long time since I performed Karnaugh map minimizations by hand. As a result, on my first pass, I missed a couple of obvious optimizations....
Nov 23, 2020
Readers of the Samtec blog know we are always talking about next-gen speed. Current channels rates are running at 56 Gbps PAM4. However, system designers are starting to look at 112 Gbps PAM4 data rates. Intuition would say that bleeding edge data rates like 112 Gbps PAM4 onl...
Nov 20, 2020
[From the last episode: We looked at neuromorphic machine learning, which is intended to act more like the brain does.] Our last topic to cover on learning (ML) is about training. We talked about supervised learning, which means we'€™re training a model based on a bunch of ...

Featured video

Synopsys and Intel Full System PCIe 5.0 Interoperability Success

Sponsored by Synopsys

This video demonstrates industry's first successful system-level PCI Express (PCIe) 5.0 interoperability between the Synopsys DesignWare Controller and PHY IP for PCIe 5.0 and Intel Xeon Scalable processor (codename Sapphire Rapids). The ecosystem can use the companies' proven solutions to accelerate development of their PCIe 5.0-based products in high-performance computing and AI applications.

More information about DesignWare IP Solutions for PCI Express

featured paper

Streamlining functional safety certification in automotive and industrial

Sponsored by Texas Instruments

Functional safety design takes rigor, documentation and time to get it right. Whether you’re designing for the factory floor or cars on the highway, this white paper explains how TI is making it easier for you to find and use its integrated circuits (ICs) in your functional safety designs.

Click here to download the whitepaper

featured chalk talk

RF Interconnect for 12G-SDI Broadcast Applications

Sponsored by Mouser Electronics and Amphenol RF

Today’s 4K and emerging 8K video standards require an enormous amount of bandwidth. And, with all that bandwidth, there are new demands on our interconnects. In this episode of Chalk Talk, Amelia Dalton chats with Mike Comer and Ron Orban of Amphenol RF about the evolution of broadcast technology and the latest interconnect solutions that are required to meet these new demands.

Click here for more information about Amphenol RF Adapters & Cable Assemblies for Broadcast