It’s widely accepted that embedded programmers don’t use agile programming methodologies. Hard deadlines mean hard commitments and none of this wishy-washy, hippie-talk, “It’ll be done when it’s done” nonsense.

Which may raise two questions in your mind. Most fundamentally, what does “agile” mean? And, perhaps more critically, is this truism actually true? There was a session covering this at DesignWest (or ESC), and it’s funny how process can get in the way of progress, even for processes that purport to value progress over process.

Let’s unpack this.

First of all, what is “agile”? Like most buzzwords, it can mean different specific things to different people. But the upshot is this: rather than amassing a massive mess of a plan for a giant software development effort, you might say that you nibble away at it, planning a little, executing a little, seeing how it goes, planning more, and so on.

It’s like internet routing. When a packet leaves your computer, it doesn’t have a map of the entire route it’s going to take to get to its destination. It knows only where it’s going next. From there, it will be decided what the “next hop” will be, and so on until it arrives. It might not take the straightest path – it might appear Brownian in its trajectory, but it gets there without having to contain an entire map of the universe.

So instead of defining the 100 critical features needed in the next software release, you pick, oh, let’s say one important one, or maybe a critical step towards a big new feature, and you implement it. Through that process, you run into the surprises that always lurk somewhere deep in your computer. Those surprises might change your course, or not. But, at least, once done with that phase, they’re not surprises anymore.

A piecewise process isn’t the only reason for approaching it this way. It’s also a fact that things change, and the intent is that an agile process is flexible and can accommodate change easily. If you have a concrete plan, then change means a failure of the plan, and there’s this tension between admitting that the plan was wrong and ignoring the fact that things have changed. Part of the agile mentality is, “Yeah, things changed; get over it. It’s OK.”

Of course, this barely begins to shave at what the phrase “agile” conjures up. It has engendered near religious fervor, with a manifesto and different denominations – XP, Scrum, Kanban, whatever – and zealous adherents that may be so obsessed with embracing a process that eschews being obsessed with process that, if you deviate one iota from their process-less process, then you’re out of the congregation. Meet the new boss, same as the old boss.

But that’s an extreme; kind of like where, if you go too far to the left politically, you end up on the right (or vice versa). It doesn’t have to be that way.

Agile methodologies have largely been embraced in the IT space, where giant enterprise applications may be underway, for example. These are software products, and they can be released when they’re ready, and they can be patched if there’s a problem.

And this is exactly why agile is viewed as poorly suited to embedded systems: embedded software doesn’t exist on its own; it serves a larger system and must be delivered in concert with that system. Yes, some more capable systems (i.e., less demanding of low power, size, or cost) may be amenable to in-field upgrades, but there’s much less of a ship-it-and-patch-it mentality. It’s got to work.

And since there’s a schedule on when the system will ship, there’s a schedule on when the software has to be ready. And it really feels more reassuring to revert to the tried-and-true waterfall approach or some other less vague scheduling technique.

But, in fact, many developers are increasing their communication and flexibility in a number of ways. You might look for some of these symptoms:

• They’re planning activities not by estimating absolute time required to get them done, but using some relative reference. I know of one team that used “gummy bears” as the standard unit. The idea is that it’s a meaningless entity in and of itself, with a nonsense name that doesn’t purport to be anything more than an arbitrary reference, but that an experienced team knows how many such units of effort will be required for a given goal. Relative accuracy is pretty good, while absolute scheduling accuracy is not so good.
• They meet regularly, perhaps daily, perhaps not, for a short period of time to assess blockers and resolve them. They may or may not stand up during the meeting.
• They may assign pairs of programmers to work together.
• They run unit tests with each change. They may have systems that automate unit testing so that each save triggers a test run.
• They integrate constantly, doing nightly builds and running regression suites with each build.

Don’t tell anyone, but these are hallmarks of agile programming. The engineers might not be wearing the right shirt or doing the right handshake or intoning the right incantation for membership in an official agile cult, but that’s ok – they’ve ignored the style in favor of the substance. Usually a good choice.

In other words, even though very few embedded development teams may be card-carrying agile adherents, they are, in fact, using agile methods. Our opening assumption may, in fact, not be true.

But proponents of more traditional approaches may ask a reasonable question: why meander around like this instead of getting right to the goal with a bit of better planning up front? Why use a Brownian trajectory when you can go straight to the destination along the shortest path? A waterfall-like schedule gives everyone the comfort that a plan is in place with deadlines and accountability. It ensures that all of the pieces of the embedded system will come together in time to meet the market window.

And, let’s face it: everyone knows that agile is just an excuse for consultants to avoid a straight line because, in wandering through the weeds, they can run straight to the bank with the cash earned by vague projects with no goals that can therefore never end. It’s a full-employment program. Real success happens by setting goals and, if they’re not met, severing heads.

Which sounds reasonable. (OK, not sure about the death penalty part, but we won’t go there.) Except for one thing: agile proponents would say that such planning is a fiction. Waterfall schemes that seem tried-and-true are really only tried; they’re almost never true. When have you ever seen a two-year project that came in on time exactly as originally planned? Especially one involving software?

There are four possible ways standard projects can go.

• Everything happens as planned.
• Best efforts aside, unexpected changes or issues screw up the plan.
• The original plan was never intended to succeed; buy-off was intended to ensure funding, and then the plan could take on a life of its own once everyone was in too deep to back out.
• The planners knew that the plan couldn’t work, but, rather than intending to take things for a ride, they knew that management had certain unrealistic expectations, and that reality wouldn’t win the argument: “If I don’t give them the schedule they want, they’ll replace me with someone that will – even though the new guy won’t be able to meet it any more than I will.”

Three out of those four outcomes ever really happen. I’ll let you guess which three. (OK, I guess I’m letting out my inner cynic here…)

So yes, in a perfect world, a solid plan would be ideal. And an internet packet would know exactly what its route would be when it left its origin – and it would be the shortest possible route. But that’s the thing: we never know exactly what lies between us and the end. Heck, in some cases, we don’t really know the end – we only think we do.

And if, in fact, you end up going off course, an agile (or agile-like) process will let you know sooner; you can make little course corrections all along, rather than charging ahead and eventually realizing that you’re miles away from where you wanted to be.

It takes serious guts for a corporate management team to fess up to the fact that they have limited knowledge of the future. Too many managers cling to the old cop out, “Give me results, not excuses.” Which is equivalent to them plugging their ears and going, “Lalalalala,” when someone tries to bring them some inconvenient reality. It’s too easy to blame your team for screwing up rather than taking ownership of uncertainty and embracing – and managing – risk rather than pretending it doesn’t exist.

But more teams are, in fact, moving towards methodologies that acknowledge and accommodate the unknown and the unexpected. They can be flexible; they can be nimble; they can be reactive.

They’re just not calling it agile.