Do you remember the first time you sat on a bicycle? The thing wobbled and slid so that you thought you would never get to ride it like the other children. Or when co-ordinating clutch and gas pedal seemed impossible? The car jumped like a kangaroo and then stopped dead. Of course, with an automatic that isn’t a problem: you just sit down and press.
Developing software is hard. So people are always looking for easier ways, looking perhaps for the software equivalent of replacing the gearbox and clutch with an automatic transmission. This need is regularly met by another new prophet declaring another new route out of the wilderness of software development. The majority of these prophets are preaching to the members of the church of the enterprise system, who have been trying to find simpler ways of developing and maintaining systems since at least the invention of COBOL (Common Business Oriented Language) in 1959.
A common problem is that programmers who follow a prophet often become fanatics – embracing the prophet’s views with fervour and denouncing all other approaches as heresy. Over the last few years, an approach that was developed for the enterprise has begun to make its appearance in the embedded field. Scrum, Agile and a range of related techniques and methods are now seen by some as providing the only true path to enlightenment. Using them will produce “better” software in a shorter time and will also produce happier people. That last promise is not an exaggeration: the true believers do promise a better life – check some of the company sites on the Internet.
Probably the best starting point is the Agile Manifesto. This has its own website, agilemanifesto.org, and as the manifesto is short, I will quote it in full:
Manifesto for Agile Software Development
We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more.
That is it. But it tells you nothing about how Agile and its surrounding framework, Scrum, actually work.
The Agile critique of a classic development project is that it starts with a requirements document, which attempts to describe in detail all the elements that are to be included in the final product. There may be no indication as to which elements are essential, which are just “nice to have,” and which have been included just in case they might be useful. It is well documented that those producing a requirements document tend to cover themselves by throwing everything they can think of into the specification.
The requirements document is passed to the development team, which disappears behind closed doors for months, if not longer, and produces software that it thinks matches the requirements.
In the meantime, life has moved on. New requirements have been identified and either fed into the development team, slowing down the development cycle, or stored for later. (Changing requirements is one of the most common causes for projects to over-run both time and budget, or even to fail). The customer looks at the software and immediately identifies areas where the requirements were imperfect, either lacking or incomplete, or where they have been wrongly interpreted or implemented. The decision is now whether to go with what has been delivered, try to fix the software, or just give up.
By contrast, the Agile acolytes will say, an Agile project starts with a meeting between a small software team and the “customer.” The intention is that this meeting should produce a list of what the software should achieve, ordered by priority or importance. The team takes the top items on the list, goes away and writes working software to achieve those objectives, producing the software within an agreed period of time, normally two or four weeks. Once the customer has accepted the software, then the next group of objectives is taken and more working software produced. (In Agile/Scrum jargon, each session of developing software is called a sprint.) This process can continue until the list of objectives is exhausted. But since it is a relatively simple task to see how much longer the project will run, the customer and the team at their regular meetings can also review the outstanding objectives and make decisions on which of those left are necessary for the completion of the project and which can be seen as “nice to have”.
There are several advantages to this approach. One is that if requirements change as the project progresses, these changes can be accommodated. The customer sees things happening and feels that they are involved in the project.
Now, at first sight, this sounds like a hacker’s paradise – just get on with programming. But for success, there has to be much more. The team has to work as a team, and part of the process is a daily progress meeting, where each team member provides the status of the task allocated at the previous meeting and commits to the next task. In the successful projects, software is written to conform to a style guide, has to be reviewed, and needs to be documented. It has to have appropriate unit tests developed and has to be built, tested, and checked in to the central control library. Software is not personal property; it is shared by the team.
Scrum and Agile have had a mixed success in the enterprise. Some organisations swear by it. Others have tried the approach and given up. There seems to be a cultural bias, and anecdotal evidence suggests that companies with a strong top-down culture find it difficult to adopt Agile techniques. One commentator on Agile also suggests that implementing Agile can expose existing problems within a company. A third reason appears to be that people have taken Agile to be an “easy” way to do projects, or as one commentator has called it, “a hacker’s charter.” When they discover that Agile requires a level of discipline and organisation to be successful, they start to look for another magic bullet that allows them to concentrate on code writing and ignore the boring stuff, like verification, testing and documentation.
For us, the real question is, “Is Agile useful in the embedded environment?” The most extreme case of an embedded application is a safety-critical system, and if Agile can be used here, then it is certainly possible that it can be used in other embedded applications.
For safety-critical applications, the big obstacle to the use of Agile is the second point of the manifesto: “Working software over comprehensive documentation.” Safety–critical applications are normally developed against safety standards, such as IEC 61508, or DO-178B. In these standards, documentation is central and is used as evidence that the project has been planned, implemented and tested to the exacting measures of the standard. A further requirement is often independent testing: a third party develops tests according to the original specification and uses them to verify that the software actually meets the specification. While neither of these is an insurmountable obstacle, adding them to an Agile project would add to both the time taken and the overall cost. Finally, an important ingredient of Agile is small teams: many safety-critical projects are very large and use equally large software teams.
Agile methods have their strengths. As a development methodology for applications other than safety-critical it probably is as valid an approach as the widely used Waterfall and V methodologies. The problem in trying to evaluate any of these is that the adherents often overstate the case for their methodology and equally overstate the case against the others.
In a few years time, yet another methodology will make its appearance, and again, a new set of debates will occur. However, nothing will get around the problem that producing robust systems that meet the requirements, however expressed, and that can be maintained and extended is hard and will remain so.
