One of the common underlying themes of great computer science
textbooks is the aesthetics of complexity, by which I do not mean the
glorification of the complicated; on the contrary, they are all about
finding the simplicity that underlies the complicated. A section of
one of these textbooks will typically start by explaining a difficult
problem, and then it will deftly reveal some abstraction or theory
that magically makes the problem simple. A really great textbook will
then leave the reader wondering what was so difficult about the
problem in the first place.

This is the essence of software engineering: decomposing an activity
into simple parts that fit together smoothly and elegantly. It is the
creation of abstractions -- data structures and algorithms, objects
and methods -- that together not only solve the problem, but also
illustrate how the solution works. Good software gives you a new
understanding that allows you to solve related problems by simply
fitting together the parts of the software in new ways; really good
software provides such powerful abstractions that the parts fit
together in ways that the original designer didn't expect, to solve
problems outside the original scope.

Bad software is like those humorous posters depicting "a day at the
beach", covered with hundreds of people doing ridiculous things.
Little bits of it may be amusing, but they combine to make a whole
that is messy and ugly. Good software is like Monet's Water Lilies,
where the basic theme of the brush strokes builds up to an attractive
whole. It's the miscellaneous collection of buildings of Oxford's
Bodleian compared to the coherent whole of the Cambridge University
Library. It's the soundtrack for a Tom & Jerry cartoon compared to a
Bach fugue.

The IOCCC is such a great joke because it is the ultimate satire on
software engineering: it glorifies the complicated, the special-
purpose, the incomprehensible. An IOCCC winner is like a Heath
Robinson mechanism in the way it employs a wonderful intricacy to
perform a trivial task. There is irony too, though, because without
using abstractions an IOCCC entry cannot accomplish enough to win; and
without elegance of composition a Heath Robinson painting cannot
illustrate the workings of its ridiculous machine.

One of the things that distinguishes 21st century programming from
programming in earlier eras is that it has become much more about
communicating with other people, rather than purely about making
computers do things. Our machines have enough capacity that we can
afford the luxury of explanation as well as operation. This is why I
think Knuth should be shunned. Although the text of the Art of
Computer Programming does a good job of discussing the mathematical
principles of algorithms and data structures, his code is stuck in the
era before structured programming. It is very hard to transcribe
either his spaghetti English or his prematurely optimised assembler
into what would nowadays be considered code of acceptable quality.
This is not just unthinking "goto considered harmful" dogma: it also
weakens the books' credibility in the areas of formal correctness and
rigour, since the mathematical tools we nowadays use to understand
code formally depend on structured programming. These are the same
tools used by compilers to optimise code, so code written to the
common best practice will also run most efficiently. He refuses to
address this deficiency in the forthcoming editions of his books,
preferring to remain defiantly 40 years out of date. I therefore
recommend that Knuth should be way down on people's shopping lists,
after books by Sedgwick and Cormen/Leiserson/Rivest and Okasaki.

In contrast, a book which does describe the right way to approach
design isn't about programming at all. Tufte's Visual Display of
Quantitative Information may be about statistical graphs, but the
principles it expounds are also principles of software engineering.
Its main themes are economy and elegance, and the rules it derives are
based on Einstein's razor "as simple as possible, but no simpler", and
using orthogonality to allow a single graphical element to perform
multiple tasks. It is thoroughly excellent: as you would expect, it is
beautiful to behold, but it is also very readable. The text has a
personality, opinionated yet modest: although the book contains many
graphs, they are all examples; when the author uses statistics to
support his argument he presents them in a table; and frequently when
he is critical of a graph it is one of his own designs. The reason
this is a book for programmers is illustrated by this quote from the
epilogue, which would require only the most trivial changes to make it
directly relevant to software engineering:

 What is to be sought in designs for the display of information is the
  clear portrayal of complexity. Not the complication of the simple;
   rather the task of the designer is to give visual access to the
   subtle and difficult -- that is, the revelation of the complex.

$dotat: doc/web/writing/complexity,v 1.3 2001/12/17 15:15:29 fanf Exp $