[tjr]

That is engineering, software projects with today's tools and techniques are not. This will happen some day in software. We are not there yet, by far.

Sure we are, at least pretty close.

Commercial avionics software developed to DO-178B standards calls for reams of requirements, verification tests, compliance to process, internal quality reviews, external audits, and sign-off by FAA representatives.

A one-line code change can take days to implement, and might not be released to "users" for months or years.

But the software is extremely robust.

If we wanted to engage in the same level of software engineering for all software, we could. But we don't want to. Developers don't want to, and users don't demand it. If an iPhone game crashes, who cares? If a productivity application crashes, you might have lost an hour's work, but it's probably not so annoying so as to warrant a couple orders of magnitude more cost associated with the software.

But if a software failure could kill people, well, that's different. It's worth spending a huge amount of time to make it perfect.

Avionics software can be so thoroughly tested because it is thoroughly designed up front. You know exactly what it's supposed to do. Much less-critical software is designed in a more ad hoc fashion; or there might not even be a design at all! How much software has been organically grown, starting with an idea and hacking on it until it seemed to work?

If you want to thoroughly test that, you have to go back and thoroughly state what it's supposed to do.

It's quite possible, but by and large it's not desired enough to make it worth actually doing. I'm not sure how this could change, or even if it should change. Instant bug fixes on web applications are cool, even though they come with the risk of having broken something else...

[cbd1984]

> But the software is extremely robust.

Does the specification specify the input as well, or is it actually robust against real input?

By real input I mean stuff like HTML tag soup: There's no single standards document which describes it fully, or even mostly, and it isn't going to be fixed. Ever. It simply has to be processed, to the limits of your ability to process it.

Avionics software is robust, sure, but it's almost a toy problem, its domain is so well-specified. You can ignore so much about reality because you've got a contract which says "We only care about what's listed, everything else can go hang" and in the real world (or, well, in the rest of the real world) you can't usually do that.

[tjr]

A fair observation. In the example of handling HTML input, I would suppose that's not a problem with individual developers, but a problem with the industry. Such a relaxed format should not have been allowed to exist, if the industry cared about its software products being as robust as possible.

I'm failing to think of any avionics application that might handle HTML, but avionics systems do have their own formats to deal with. ARINC 661, for example, is an XML file format for transmitting graphical display elements:

http://en.wikipedia.org/wiki/ARINC_661

Of course, all uses of ARINC 661 data are thoroughly tested. I'm not sure I would go so far as to describe it as a "toy problem", but it certainly does intentionally limit the problem domain to exactly what needs to be dealt with. Malformed ARINC 661 data received would just be discarded, not tried to be displayed in the best possible way even if it wasn't quite right, because that would be unacceptable; the problem would be with whoever was sending the malformed data.

Anyway, you're quite right though; without a precise and unambiguous format definition, you can only go so far down the path of robustness.

[jasonm23]

> Such a relaxed format should not have been allowed to exist.

Amen

[graycat]

Yes, long ago it was observed that the first step in proving software correct was a clear specification of what the software was supposed to do and that for a lot of realistic software writing that spec was unreasonably difficult.

[bstpierre]

> If we wanted to engage in the same level of software engineering for all software, we could. But we don't want to.

Of course. Like building structures, there are different kinds of software. I can build a garden shed by myself as long as I have the skills to get the thing to stand up. If it blows down in a storm I'm the only person with a loss. But just because I can get a garden shed (or even a larger structure like a barn or a house) to survive windstorms without incident doesn't make me a structural engineer.

[jng]

In my understanding, those projects are the modern pyramids of software. Built by sheer brute force at an unsustainable cost, only affordable by a select few.

Large amounts of reliable, performant and scalable software will be built on time and on budget at some point in the future, with a cost and effort similar to today's run-of-the-mill software development. This will happen when, thanks to better understanding of the principles, we can create better tools and techniques to do so.

[tjr]

I think sheer brute force would be far less organized. How do you suppose designing and constructing a building according to designs and building codes is a more advanced, less brute-forced activity than building software according to requirements and industry standards?

But avionics-style software engineering need not be an all-or-nothing approach. Elements of it could be introduced into other programming applications for increased robustness. Greenspun wrote up an excellent article on adding external design review to web application development:

http://philip.greenspun.com/software/design-review

Such would not be a heavy burden on a project, and would likely help catch at least the most glaring errors that went unnoticed by the developers.

In any event, I agree that better tools and techniques offer the tantalizing possibility to help all software be more robust, even if it is never more "engineered". Modern languages have, for example, done away with whole categories of bugs that used to plague C programmers (and still do, unfortunately).

[jng]

"Sheer brute force" is probably a bit of an exaggeration. But just a bit. As you describe yourself, the kind of advance from C to more modern languages is a step away from "sheer brute force" and towards reasonable approaches. And only a small step compared to where we have to get before this is engineering.

I'd say a much larger part of all software projects is dysfunctional compared to the same in architectural or civil engineering projects, and I think this situation will greatly improve in the future, thanks to a handful of qualitatively innovative insights providing enormous improvements that we are yet to see.