I believe we are still in the alchemical age of software application design. There are glimmers of enlightenment here and there that will help the transition to chemistry, but the transition hasn't really begun. Software done well today requires genius, but this should not be, just as basic chemistry labs don't require it.
Djikstra was complaining about these alchemical methods, and rightly so. The sad and dangerous thing is how much we have built on the shaky edifice of alchemy, and how much we blame ourselves for failure, and how we pride ourselves when we master some local improvement. (Even worse, the pride of finding some new local improvement).
"He talks about using OOP to remove "accidental difficulty" from the process, “allowing the designer to express the essence of the design without having to express large amounts of syntactic material that add no information content”…
"…but he stops at OOP? Makes me think that he was not very informed about functional programming, given that OOP is (1) trivial to add to functional programming languages, (2) dissed by all the greats as uninteresting – or worse."
Here's my take: OO may be a decent way of managing complexity, but it introduces a (noticably high) complexity floor that negates many of the benefits. It works very well for some things, specifically windowed guis, and less well for almost everything else. Rule of thumb: if you are modeling the "problem domain" and not the "solution domain," you are doing it wrong.
However, for better or worse, every programmer in the last 30 years has been trained on OOP; most simply cannot conceive of not using a "class" keyword. On the other hand, almost no one does OOP itself anymore. Objects, classes, are treated as a module system, which is noticably missing, or at least very limited, in most programing environments.
All problems have some level of essential complexity, although often it is lower than both the floor demanded of both the programing environment and the programmer's self esteem. (It's difficult for many to admit that what they are being asked to do is trivial, so they look for ways to make it more difficult, including introducing bizarre monstrosities into their environment.)
On the other hand, exactly no one wants to grapple in hand to hand combat with any moderately complex problem (and many problems do have sudden, steep, complexities) as Dijkstra would have preferred. Or, some problems cannot be solved in the necessary resource constraints, including the brain power of the programmer. As a result, more complexity is pushed down stream---and modern systems have very long tributaries.
Hmm...I'd say Smalltalk is about as simple as things get.
For anything more than a few functions/methods, you want some sort of scoping/module concept, and I think you're right that that's one way classes are used. Might as well be classes as they are lightweight, provide a convenient metaphor and other benefits.
I would say that true OO programming is rare. Most I've seen in industry is scoped procedural with little to no polymorphism. Or paraphrasing Alan Kay: "Before you dismiss OO, you should at least try it once".
Smalltalk is simple. But Smalltalk doesn't help making your programs simpler - at least not as much as a powerful FP language.
OOP is a convenient tool for cutting your state into smaller, independent pieces. But that can only go as far as the pieces can be independent, what for most practical domains is not very much. It also does not bring any other big benefit bundled with it, what looks to me like the EDJ complaint is all about.
> But Smalltalk doesn't help making your programs simpler
Hmm..."OO may be a decent way of managing complexity..." was what the parent said, I responded to the "complex baseline" assertion. And your assertion that it doesn't help is empirically invalidated on many counts.
First, the folks at Xerox PARC managed to implement "Personal Computing (networked)" in around 20KLOC of Smalltalk, including the OS, windowing system, IDE, networking, e-mail, messaging etc. Second, look at the sheer amount of functionality in something like Squeak.
> at least not as much as a powerful FP language.
Citation needed? Caper Jones Function Point Metric puts Smalltalk's language level at 20, Haskell at 8.5 (larger=better)
I found this interesting, so after some googling, these are the numbers I found:
QNX kernel = 23 KLOC, C [1]
Plan9 kernel = 50 KLOC, C [2]
Squeak VM = 9 KLOC, Smalltalk [3] (Squeak is a Smalltalk)
Note, those are the kernels - Smalltalk's 20 KLOC include the whole system (GUI, etc), IIRC.
For clarity, I'm a bit out of my depth here, and I'm assuming that the Squeak VM is a bit akin to a kernel. And surely one could find more authoritative references for it all.
The ~20KLOC of Xerox PARC was the entire system, not just the kernel, and included, among other things:
- the kernel/OS
- the windowing GUI
- the programming language, compiler, libraries
- the visual IDE
- Networking
- E-Mail/Chat
- ...
I am not sure DTP/Word Processing was also included, it might have been a separate system.
Now I am a huge fan of QNX, used it and developed for it a bit in the 90s, but the version that fit on a floppy didn't cover anywhere near as much. Also, QNX is built on a message passing kernel...hmm...
You also left out Amiga Exec + Intuition, which is also amazingly compact and was built in C...but using OO concepts, which are implementable in C. For example, everything in the kernel is a List, and so specific concepts like ports effectively subclass List (with function tables etc.) See also: Objective-C.
Plan 9 is also very, very cool, and its "everything is a file" (no really, everything) is not that dissimilar to everything is an object. Part of my own work[1] is trying to find the common ground between these two approaches, with a good amount of success.
Anyway, the Plan 9 distribution is 1.5 million lines of code. The Acme text editor and cc compiler alone sum to 20KLOC. libdraw is another 5KLOC etc.
So Plan 9 is good, Smalltalk is arguably better, but I think we can (and have to) do better still, see VPRI[2].
Coming back to Brooks for a second: in NSB, he saw OO as one of the few hopes for tackling the problem, and in an OOPSLA panel 20 years later he opined that it had largely succeeded[3].
That said, I agree with your take that software tends to be much, much more complicated than it has any need or rights to be. I attribute much of this to architectural mismatch[4], particularly in the domain of UI-intensive systems[5]
For example, much of the complexity of UI programs can be removed by modeling them using high-level dataflow constraints[6], combined with compositional storage.
Dataflow also seems to be a very powerful reduction factor, Unix pipes/filters still rule (see Doug McIllroy's response to Bentley's challenge), and Dan Amelang's Nile/Gezira does a modern 2 1/2D compositing graphics subsystem in ~500 lines of dataflow code[7]
Clickbait. The post does not contrast Fred Brooks with Dijkstra.
(Instead it presents some complaints from Dijkstra - on overblown marketing material mostly. And presents some of Fred Brooks well-known points - apparently the author doesn't quite agree, or perhaps hadn't grasped the depth of Brooks' thinking.)
The post quotes both Brooks and Dijkstra; the first managed development of a system and got famous, the second saw the system's actual failures and reported his criticism on it.
So, care to explain what exactly is it that you find lacking? My experience with people who cite "failure to grasp the thinking" has been rather with "architecty" types that can't get past handwaving to make a point, so anything concrete would be appreciated.
For example: you say Dijkstra complains "on overblown marketing material mostly". Citation, please? Sounds like you mean that the marketing overpromised and underdelivered, but to me it looks like Dijkstra was pretty clear about the whole set of ideas being poppycock even if fully delivered; he says that it's fundamentally the wrong way to do things, and does so in a rather smug "told you so" mode.
... unless we want to argue that maybe Dijkstra didn't "grasp the depth of Brooks' thinking" either, nor the functionality of that thinking's actual, first hand implementation?
Djikstra was complaining about these alchemical methods, and rightly so. The sad and dangerous thing is how much we have built on the shaky edifice of alchemy, and how much we blame ourselves for failure, and how we pride ourselves when we master some local improvement. (Even worse, the pride of finding some new local improvement).