"No Silver Bullet - Essence and Accident in Software Engineering - Frederick P. Brooks"

ADVERTISEMENT
ADVERTISEMENT

Download "No Silver Bullet - Essence and Accident in Software Engineering - Frederick P. Brooks"

Download PDF

Fill PDF online

Rate (4.7 / 5) 32 votes
No Silver Bullet
—Essence and Accident in Software Engineering
Frederick P. Brooks, Jr.
University of North Carolina at Chapel Hill
There is no single development, in either technology or management
technique, which by itself promises even one order-of-magnitude
improvement within a decade in productivity, in reliability, in simplicity.
1
Abstract
All software construction involves essential tasks, the fashioning of the complex
conceptual structures that compose the abstract software entity, and accidental tasks, the
representation of these abstract entities in programming languages and the mapping of
these onto machine languages within space and speed constraints. Most of the big past
gains in software productivity have come from removing artificial barriers that have
made the accidental tasks inordinately hard, such as severe hardware constraints,
awkward programming languages, lack of machine time. How much of what software
engineers now do is still devoted to the accidental, as opposed to the essential? Unless it
is more than 9/10 of all effort, shrinking all the accidental activities to zero time will not
give an order of magnitude improvement.
Therefore it appears that the time has come to address the essential parts of the
software task, those concerned with fashioning abstract conceptual structures of great
complexity. I suggest:
Exploiting the mass market to avoid constructing what can be bought.
Using rapid prototyping as part of a planned iteration in establishing software
requirements.
Growing software organically, adding more and more function to systems as they are
run, used, and tested.
Identifying and developing the great conceptual designers of the rising generation.
Introduction
Of all the monsters who fill the nightmares of our folklore, none terrify more than
werewolves, because they transform unexpectedly from the familiar into horrors. For
these, we seek bullets of silver than can magically lay them to rest.
1
Reproduced from: Frederick P. Brooks, The Mythical Man-Month, Anniversary edition with 4 new
chapters, Addison-Wesley (1995), itself reprinted from the Proceedings of the IFIP Tenth World
Computing Conference, H.-J. Kugler, ed., Elsevier Science B.V., Amsterdam, NL (1986) pp. 1069-76.
No Silver Bullet
—Essence and Accident in Software Engineering
Frederick P. Brooks, Jr.
University of North Carolina at Chapel Hill
There is no single development, in either technology or management
technique, which by itself promises even one order-of-magnitude
improvement within a decade in productivity, in reliability, in simplicity.
1
Abstract
All software construction involves essential tasks, the fashioning of the complex
conceptual structures that compose the abstract software entity, and accidental tasks, the
representation of these abstract entities in programming languages and the mapping of
these onto machine languages within space and speed constraints. Most of the big past
gains in software productivity have come from removing artificial barriers that have
made the accidental tasks inordinately hard, such as severe hardware constraints,
awkward programming languages, lack of machine time. How much of what software
engineers now do is still devoted to the accidental, as opposed to the essential? Unless it
is more than 9/10 of all effort, shrinking all the accidental activities to zero time will not
give an order of magnitude improvement.
Therefore it appears that the time has come to address the essential parts of the
software task, those concerned with fashioning abstract conceptual structures of great
complexity. I suggest:
Exploiting the mass market to avoid constructing what can be bought.
Using rapid prototyping as part of a planned iteration in establishing software
requirements.
Growing software organically, adding more and more function to systems as they are
run, used, and tested.
Identifying and developing the great conceptual designers of the rising generation.
Introduction
Of all the monsters who fill the nightmares of our folklore, none terrify more than
werewolves, because they transform unexpectedly from the familiar into horrors. For
these, we seek bullets of silver than can magically lay them to rest.
1
Reproduced from: Frederick P. Brooks, The Mythical Man-Month, Anniversary edition with 4 new
chapters, Addison-Wesley (1995), itself reprinted from the Proceedings of the IFIP Tenth World
Computing Conference, H.-J. Kugler, ed., Elsevier Science B.V., Amsterdam, NL (1986) pp. 1069-76.
F. Brooks: No Silver Bullet—Essence and accident in software engineering (1986)
2
The familiar software project has something of this character (at least as seen by the
non-technical manager), usually innocent and straightforward, but capable of becoming a
monster of missed schedules, blown budgets, and flawed products. So we hear desperate
cries for a silver bullet, something to make software costs drop as rapidly as computer
hardware costs do.
But, as we look to the horizon of a decade hence, we see no silver bullet. There is no
single development, in either technology or management technique, which by itself
promises even one order of magnitude improvement in productivity, in reliability, in
simplicity. In this chapter we shall try to see why, but examining both the nature of the
software problem and the properties of the bullets proposed.
Skepticism is not pessimism, however. Although we see no startling breakthroughs,
and indeed, believe such to be inconsistent with the nature of software, many
encouraging innovations are under way. A disciplined, consistent effort to develop,
propagate, and exploit them should indeed yield an order-of-magnitude improvement.
There is no royal road, but there is a road.
The first step toward the management of disease was replacement of demon theories
and humours theories by the germ theory. That very step, the beginning of hope, in itself
dashed all hopes of magical solutions. It told workers the progress would be made
stepwise, at great effort, and that a persistent, unremitting care would have to be paid to a
discipline of cleanliness. So it is with software engineering today.
Does It Have To Be Hard? – Essential Difficulties
Not only are there no silver bullets now in view, the very nature of software makes it
unlikely that there will be any−no inventions that will do for software productivity,
reliability, and simplicity what electronics, transistors, and large-scale integration did for
computer hardware. We cannot expect ever to see twofold gains every two years.
First, we must observe that the anomaly is not that software progress is so slow but
that computer hardware progress is so fast. No other technology since civilization began
has seen six orders of magnitude price-performance gain in 30 years. In no other
technology can one choose to take the gain in either improved performance or in reduced
costs. These gains flow from the transformation of computer manufacture from an
assembly industry into a process industry.
Second, to see what rate of progress we can expect in software technology, let us
examine its difficulties. Following Aristotle, I divide them into essence−the difficulties
inherent in the nature of the software−and accidents−those difficulties that today attend
its production but that are not inherent.
The accidents I discuss in the next section. First let us consider the essence.
The essence of a software entity is a construct of interlocking concepts: data sets,
relationships among data items, algorithms, and invocations of functions. This essence is
abstract, in that the conceptual construct is the same under many different
representations. It is nonetheless highly precise and richly detailed.
I believe the hard part of building software to be the specification, design, and testing
of this conceptual construct, not the labor of representing it and testing the fidelity of the
representation. We still make syntax errors, to be sure; but they are fuzz compared to the
conceptual errors in most systems.
F. Brooks: No Silver Bullet—Essence and accident in software engineering (1986)
3
If this is true, building software will always be hard. There is inherently no silver
bullet.
Let us consider the inherent properties of this irreducible essence of modern software
systems: complexity, conformity, changeability, and invisibility.
Complexity. Software entities are more complex for their size than perhaps any other
human construct, because no two parts are alike (at least above the statement level). If
they are, we make the two similar parts into one, a subroutine, open or closed. In this
respect software systems differ profoundly from computers, buildings, or automobiles,
where repeated elements abound.
Digital computers are themselves more complex than most things people build; they
have very large numbers of states. This makes conceiving, describing, and testing them
hard. Software systems have orders of magnitude more states than computers do.
Likewise, a scaling-up of a software entity is not merely a repetition of the same
elements in larger size; it is necessarily an increase in the number of different elements.
In most cases, the elements interact with each other in some nonlinear fashion, and the
complexity of the whole increases much more than linearly.
The complexity of software is in essential property, not an accidental one. Hence
descriptions of a software entity that abstract away its complexity often abstract away its
essence. Mathematics and the physical sciences made great strides for three centuries by
constructing simplified models of complex phenomena, deriving properties from the
models, and verifying those properties experimentally.
This worked because the
complexities ignored in the models were not the essential properties of the phenomena. It
does not work when the complexities are the essence.
Many of the classical problems of developing software products derived from this
essential complexity and its nonlinear increased with size. From the complexity comes
the difficulty of communication among team members, which leads to product flaws, cost
overruns, schedule delays. From the complexity comes the difficulty of enumerating,
much less understanding, all the possible states of the program, and from that comes the
unreliability. From the complexity of the functions comes the difficulty of invoking those
functions, which makes programs hard to use. From complexity of structure comes the
difficulty of extending programs to new functions without creating side effects. From the
complexity of structure comes the unvisualized state that that constitute security
trapdoors.
Not only technical problems but management problems as well come from the
complexity. This complexity makes overview hard, thus impeding conceptual integrity.
It makes it hard to find and control all the loose ends. It creates the tremendous learning
and understanding burden that makes personnel turnover a disaster.
Conformity. Software people are not alone in facing complexity. Physics deals with
terribly complex objects even at the “fundamental” particle level. The physicist labors
on, however, in a firm faith that there are unifying principles to be found, whether in
quarks or in unified field theories.
Einstein repeatedly argued that there must be
simplified explanations of nature, because God is not capricious or arbitrary.
No such faith comforts the software engineer. Much of the complexity he must
master is arbitrary complexity, forced without rhyme or reason by the many human
F. Brooks: No Silver Bullet—Essence and accident in software engineering (1986)
4
institutions and systems to which his interfaces must confirm. These differ from interface
to interface, and from time to time, not because of necessity but only because they were
designed by different people, rather than by God.
In many cases the software must confirm because it has most recently come to the
scene. In others it must conform because it is perceived as the most conformable. But in
all cases, much complexity comes from conformation to other interfaces; this cannot be
simplified out by any redesign of the software alone.
Changeability. The software entity is constantly subject to pressures for change. Of
course, so are buildings, cars, and computers. But manufactured things are infrequently
changed after manufacture; they are superseded by later models, or essential changes are
incorporated in later serial-number copies of the same basic design.
Callbacks of
automobiles are really quite infrequent; field changes of computers somewhat less so.
Both are much less frequent than modifications to fielded software.
Partly this is because the software in a system embodies its function, and the function
is the part that most feels the pressures of change. Partly it is because software can be
changed more easily−it is pure thought-stuff, infinitely malleable. Buildings do in fact
get changed, but the high costs of change, understood by all, serve to dampen the whim
of the changers.
All successful software gets changed. Two processes are at work. As a software
product is found to be useful, people try it in new cases at the edge of, or beyond, the
original domain. The pressures for extended function come chiefly from users who like
the basic function and invent new uses for it.
Second, successful software also survives beyond the normal life of the machine
vehicle for which it is first written. If not new computers, then at least new disks, new
displays, new printers come along; and the software must be conformed to its new
vehicles of opportunity.
In short, the software product is embedded in a cultural matrix of applications, users,
laws, and machine vehicles. These all change continually, and their changes inexorably
force change upon the software product.
Invisibility.
Software is invisible and unvisualizable. Geometric abstractions are
powerful tools. The floor plan of a building helps both architect and client evaluate
spaces, traffic flows, and views. Contradictions become obvious, omissions can be
caught.
Scale drawings of mechanical parts and stick-figure models of molecules,
although abstractions, serve the same purpose. A geometric reality is captured in a
geometric abstraction.
The reality of software is not inherently embedded in space. Hence it has no ready
geometric representation in the way that land has maps, silicon chips have diagrams,
computers have connectivity schematics. As soon as we attempt to diagram software
structure, we find it to constitute not one, but several, general directed graphs,
superimposed one upon another. The several graphs may represent the flow of control,
the flow of data, patterns of dependency, time sequence, name-space relationships. These
are usually not even planar, much less hierarchical.
Indeed, one of the ways of
F. Brooks: No Silver Bullet—Essence and accident in software engineering (1986)
5
establishing conceptual control over such structure is to enforce link cutting until one or
2
more of the graphs becomes hierarchical.
In spite of progress in restricting and simplifying the structures of software, they
remain inherently unvisualizable, thus depriving the mind of some of its most powerful
conceptual tools. This lack not only impedes the process of design within one mind, it
severely hinders communication among minds.
Past Breakthroughs Solved Accidental Difficulties
If we examine the three steps in software technology that have been most fruitful in the
past, we discover that each attacked a different major difficulty in building software, but
they have been the accidental, not the essential, difficulties. We can also see the natural
limits to the extrapolation of each such attack.
High-level languages.
Surely the most powerful stroke for software productivity,
reliability, and simplicity has been the progressive use of high-level languages for
programming. Most observers credit that development with at least a factor of five in
productivity, and with concomitant gains in reliability, simplicity, and comprehensibility.
What does a high-level language accomplish? It frees a program from much of its
accidental complexity.
An abstract program consists of conceptual constructs:
operations, data types, sequences, and communication. The concrete machine program is
concerned with bits, registers, conditions, branches, channels, disks, and such. To the
extent that the high-level language embodies the constructs wanted in the abstract
program and avoids all lower ones, it eliminates a whole level of complexity that was
never inherent in the program at all.
The most a high-level language can do is to furnish all the constructs the programmer
imagines in the abstract program. To be sure, the level of our sophistication in thinking
about data structures, data types, and operations is steadily rising, but at an ever-
decreasing rate.
And language development approaches closer and closer to the
sophistication of users.
Moreover, at some point the elaboration of a high-level language becomes a burden
that increases, not reduces, the intellectual task of the user who rarely uses the esoteric
constructs.
Time-sharing. Most observers credit time-sharing with a major improvement in the
productivity of programmers and in the quality of their product, although not so large as
that brought by high-level languages.
Time-sharing attacks a distinctly different difficulty.
Time-sharing preserves
immediacy, and hence enables us to maintain an overview of complexity. The slow
turnaround of batch programming means that we inevitably forget the minutiae, if not the
very thrust, of what we were thinking when we stopped programming and called for
compilation and execution. This interruption of consciousness is costly in time, for we
must refresh. The most serious effect may well be the decay of grasp of all that is going
on in a complex system.
2
Parnas, D.L., “Designing software for ease of extension and contraction,” IEEE Trans. on SE, 5, 2
(March, 1979), pp. 12-138.