[sowhatquestion]

> It always amazed me when someone looks at computer systems of the 70's through the lens of "today's" technology and then projects a failure of imagination on the part of those engineers back in the 70's

True enough, but as a younger programmer, I find it pretty reasonable to look back at computer systems of the 70s and wonder if we can do better today. I feel a little bit gross every time I have to write a bash shell script (or edit config files that aren't JSON/XML/YAML, for that matter), and I don't think that's a bad impulse. That something so inelegant and unsafe is still in widespread use in 2017 really ought to be a scandal. Even if the author didn't frame the issue in the most charitable way for the earlier trailblazing generations, he's calling attention to the right issues.

In other words, if you couldn't justify something being designed a certain way de novo, why be content with the existing design!?

[onion2k]

JSON doesn't have comments so it's a bad choice for human-editable config. YAML doesn't have an end marker so you can never be sure if you've got the entire file. XML is a huge pain to edit by hand if the schema is complicated, and overly verbose if it isn't. None of them are even close to being safe (for example https://arp242.net/weblog/yaml_probably_not_so_great_after_a...). All of those choices fail your "elegance" test.

TOML is my preferred config file language option where I have a choice - https://github.com/toml-lang/toml - but I suspect that suffers a lot of the same problems.

[rendaw]

I will capitalize on this derailment to promote luxem, my flexible and minimal JSON alternative: https://github.com/rendaw/luxem#what-is-luxem

[vertex-four]

The issue with all of these, of course, is that in order to get a system running you have to configure multiple "independent" tools, processes and daemons. Think setting up a web application - you have to configure the web application to listen on a certain port/UNIX socket, then configure your web server to go find it. You then need to scale this up across logical servers separated by a network - your web servers need to communicate with your database, they need some sort of authentication key/password, etc etc. You're never just configuring one thing.

The modern solution would be that there needs to be a network configuration tool which generates specific configurations for each component, is capable of encoding arbitrary invariants, and works consistently. Configuration also needs to be "push" based on events - when a DNS server dies, it should be able to figure out "we need at least 2 DNS servers, we have 1, fire up a new one - then update all systems to know about the new one".

Configuration management systems for Linux, by and large, suck. They're very good at starting from an Ubuntu Server install and building on that, and then get more and more fragile as the system lives on. Some of them (Saltstack, for example) do have some degree of event management - you can run certain commands on certain things happening, but it's not declarative or reactive in the way you'd hope - e.g. you can't just say "this system knows about all DNS servers" and expect it to work. The Docker/Kubernetes ecosystems claim to solve the network configuration problem (in a really awkward roundabout way), but not really intra-system configuration, and it still takes a lot of manual work.

NixOS gets a lot closer - but it needs to be expanded with a constraints solver and an event processing system. It's Turing-complete, so you can encode pretty much whatever you want into it, while still being a reasonable configuration language (basic use of it looks a lot like JSON).

But the point is - the formats individual components use for configuration should be more-or-less irrelevant. They could be completely opaque, so long as it's possible to get from the network config to the individual component's config and it's possible to update that config on-the-fly. In fact, it'd be more useful to standardise on one library which can handle all that for you.

[zzzcpan]

I agree with most of your post, but this is still more complex than just adding a constraints solver and an event processing system. Different things not just depend on each other, they also require different strategies for dealing with failures. Trying to squeeze everything into a single model will not work well. Maybe something like supervision trees for services might solve that, where supervisors for each service are part of the package and handle everything from automatic configuration to failures in any way they need.

[vince14]

StrictYAML - https://github.com/crdoconnor/strictyaml

[marcoms]

Still does not allow tabs for indentation - same problem as `make` but inverted

[krapp]

>JSON doesn't have comments so it's a bad choice for human-editable config (...)

What about Lua tables?

[etiene]

I second this, I love using Lua tables as a data description language.

[Sunset]

Just add comments to JSON, Douglas Crockford can eat his heart out.

[gens]

>True enough, but as a younger programmer, I find it pretty reasonable to look back at computer systems of the 70s and wonder if we can do better today. I feel a little bit gross every time I have to write a bash shell script (or edit config files that aren't JSON/XML/YAML, for that matter), and I don't think that's a bad impulse. That something so inelegant and unsafe is still in widespread use in 2017 really ought to be a scandal.

As a relatively young programmer i look at the computer systems of the 70's and think to myself "what the fuck are programmers thinking when writing today's programs". I mean we got amazing tools and a plethora of amazing languages, an insane amount of memory and an idiotic amount of processing power, chips specialized for sound and even ones for visuals. And yet everything lags.

As for shell scripts, they are fine. Not that you have to use them, as you can use python, scheme, anything (even C with tcc).

As for XML... oh god.. it is the worst, the absolute worst, way to do config files. Jesus help, is it awful. Bad for the user, bad for the processor. Simple key–value pairs (as in "power_level: 9000") are good enough for almost all configuration. Hell even the INI format is faaaaaaaaaaaaaaaaaaaaaar better then XML (and YAML, JSON (although these are themselves better then XML)).

I look at the 70's and think "haven't we learned any fucking thing ?".

PS On topic, the Unix philosophy is still valid. For a plethora of reasons. I typed too much already but if anyone's interested in my "opinion" i'l write it later today. PPS The author of this piece doesn't even know what "the Unix philosophy" exactly is.

[pjc50]

XML is nasty, but at least it's nasty in standardised ways. I've definitely worked with worse (e.g. Cadence DEF files).

The lag should be regarded as intolerable, but instead it's routine. I shouldn't have to wait for computers like this.

Shell is a nice idea, but UNIX should either have banned whitespace in filenames or made the shell handle it properly. It's amazing how many things blow up on encountering a space, and you can upset people even further by putting newlines in your file names.

[gens]

ftp://ftp.slackware.org.uk/slackware/slackware64-14.2/ChangeLog.txt

Just ctrl+f for "xml". (edit: nvm, only 3 reports are bugs. although the libxml2 report has 3 somewhat serious bugs)

"XML is a well established standard" is usually the reason people choose XML for anything. I guess i would add "Nobody got fired for choosing XML" to the list of such sayings, if i had such a list.

[stillsut]

> what the fuck are programmers thinking when writing today's programs

The difference is in the old days, you adapted to computer. Now, computer must adapt to you.

Take a look at MS's latest text-editor program's release notes (https://code.visualstudio.com/updates/v1_10). They have to build a program that accommodates so many different existing must-have features (from a potential users POV), and for each feature, accommodate almost every flavor of configuration that has ever existed for any other text editor whose style the potential user is now "locked-in to". Otherwise user will refuse to adopt.

In the 70's, I assume the audience was a pretty tight knit community of computer engineers, coming from a time of very frustrating tooling where it wouldn't be uncommon to lose a day's work because the 'Save' function failed or something like that. In this environment, you learned the crazy unintuitive keymap for vi and called it a blessing.

Now, you yell at your editor if it doesn't remind you to save your work when you close it. Or you yell at it when it does remind you; too many pop-ups! The point is that user preferences are more fragmented and expectations are higher for features, and willingness to learn details and gotchas of a program are lower, which has created the bloated program for the simple things like text editors.

[gens]

As a contrast, I use sublime. It has (almost) everything I desire. And it saves everything i write, even in the case of a power outage.

Another example would be Acme[0].

>The point is that user preferences are more fragmented and expectations are higher for features, and willingness to learn details and gotchas of a program are lower, which has created the bloated program for the simple things like text editors.

I agree. People are people and people will always be subjective. But there should be a distinction between feature bloat and node.js bloat. Still, ofc, to each his own. (I measure how good it is for me by how much hair i pull out by using it, be it by lack of features, bugs, performance, or anything)

[0]https://www.youtube.com/watch?v=dP1xVpMPn8M and https://www.usenix.org/legacy/publications/library/proceedin...

[shakna]

This might sound odd.

Instead of writing bash scripts, learn awk.

Awk is old, much older than bash, and has largely gone unchanged since '89.

But, despite the fact it was born in an era where we were still nailing down PLT and what was important in a language, it ticks a lot of boxes.

Having written some sizeable (5000+ LOC) programs in it, it kind of feels like a dynamic mash up of C, and Lisp.

Try Awk, and see if quick scripts are still as painful.

[jnunoferreira]

Yup, XML is ugly to write by hand, but I'd argue it is nearly as legible as YAML/JSON, with the great-great advantage of being able to be structurally validated.

[rbanffy]

> wonder if we can do better today

Cough... systems... cough...

We can do better. Plan 9 existed. Apple's Newton has a unique take on data files. AS/400's map all storage to memory addresses giving a single address space for everything. With Smalltalk, applications would just be new classes you import into your system.

The key to Unix's success seems to be not doing better, but doing as little as possible while still remaining useful.

And yes, it's a bit disappointing that, when we did better, we were less successful. Today, all the dominant OSs are children of either Unix or VMS.

And MVS, which went nowhere, but in the sense it's always been there and, it seems, always will be.

[LordKano]

I understand why they wonder but it should be a 10 minute thought exercise in most cases.

You have to define "better". Is it synonymous with "easier", "type safe", "less resource intensive", "object oriented", "functional", "faster" or some combination thereof.

How will you maintain backwards compatibility? Because, despite the fact that you may have just graduated, there are scripts and programs that rely on scripts that go back decades and people are NOT going to rewrite them just because of your new whiz-bang solution.

You may use it and if you represent a large enough organization, it might be viable for that use case but if you're only shopping a solution in search of a problem, you may as well not waste your time.

[marcosdumay]

Plan 9 is not "better enough". The other ones had fundamental non-technical problems that hurt their adoption (Plan 9 had those at the beginning too).

Back to Plan 9, you can expect something that is just slightly better than your current options, and that keeps being this way to steadily gain adoption until it's popular. But that requires it being no worse than the current on any popular use case. Plan 9 may have a great architecture (I'm not sure about that anymore), but OSes compete on many different fronts, like portability, performance, software availability...

If the Plan 9 architecture were something amazingly better that solved a lot of problems, you could be sure some people would have adopted it, and then it would start improving on the other dimensions. But it's not. It's better. It will make your life easier. Yet it will bring no order of magnitude change. So, people simply take a deep breath and write (again and again) the 3 or 4 lines of code required to solve their problem in Unix, instead of changing everything.

[gkya]

I'll say that if you could run a web browser on Plan 9 and maybe there was a POSIX or a Linux compat layer, it could've become popular. It'd be a boost to porting efforts if it was a usable desktop OS. Give me the best OS, but if I have to write a whole desktop/server stack on it to make use of it, I'll stick with what I have at hand.

A plethora of software is available for Unix and it's why it's here still. Had I the chance to switch to, say HaikuOS, without a big/complete productivity loss, I would.

[spyrosg]

Your stated reasons aren't valid. I think they ported Mozilla until 2002 (Russ Cox, maybe?). There was also a compat layer: APE.

As a counter to your argument: a system that has little capability compared to the well-established alternatives, starts from nothing, yet takes over the world: Linux.

I really believe there was no greater reason for Plan 9's failure to catch on (or at least it's ideas) than my peers not being interested in their own education.

[sliken]

I read the plan-9 papers and was excited to play with it. I had a .edu affiliation and it was still $500 for a tap eand license and a letter saying what you planned to do with it... or some such. I had no budget, so we used linux.

They did open it up eventually, after linux had quite the following. It it had been open/available it would have had a chance.

[spyrosg]

It sounded fine the first time I heard it, but I don't buy the "it wasn't free and Linux was available" argument any more. If more Linux users had read the papers like you, and done so early enough, they might have had an aggregate influence on Linux's design when it was still malleable. A Plan9-like Linux would have been just as good as Plan9 eventually.

[zzzcpan]

Plan 9 had fundamental technical problems too. Like most of the systems of that time it was designed with small single-building kind of networks in mind and didn't offer any value once large planet scale networks came to life. Also everything could have being implemented on top of any unix system, but wasn't that interesting for anyone to bother.

[spyrosg]

Didn't Plan 9 eventually get some mechanisms to facilitate interactions between such local-only networks around the world? 9grid and such.

In any case, Plan 9 is still ahead on quite a few fronts, with federated authentication that is simple(-ish) and works, a versioned filesystem by default, a unified namespace (the new interpretation of "filesystem") to access any resource you might want to use (instead of, today: DNS + TCP ports + HTTP URL's), and security by only exposing the necessary resources to processes, instead of exposing everything and then making up contrived access restriction schemes on top. (Smells of capabilities by Pike's own admission, I believe). That last part was mostly thanks to namespaces which Linux has picked up and popularised in a simple form with Docker... after 20+ years.

The important bits of the new architecture were also about the parts that were left out. No disk usage utility, based on the rapidly decreasing price of a giga bytes. Go and try to make contemporary sysadmins, who still partition their drives to this day, accept this highly distruptive simplification. Gotta create ourselves our busywork!

In the end, Plan 9 was made by /educated/ people with /resources/, and much of today's software isn't. No surprise it's better made.

(edit: some structure added to my rambling.)

[pjmlp]

Most of them were fixed in Inferno, but by then most stop paying attention.

To the point even on HN many seem to be unaware that the line ended on Inferno and not Plan9.

[cronjobber]

Personally, I tend to slot Inferno closer to Java than to Plan9, even though I'm clear about the lineage. In that sense, the line of "real" operating systems ended at Plan9.

[pjmlp]

For me it is Inferno.

If you look at the architecture of mobile OSes, they were in the right direction.

[kbart]

"if you couldn't justify something being designed a certain way de novo, why be content with the existing design!?"

Please, no. Don't convert system programming/engineering to the mess that is web development today.

[madmulita]

I'm sorry, Docker is already here, even in enterprisie flavor.

[bad_user]

> why be content with the existing design!?

Because backwards compatibility is more important. We can't just throw everything away every couple of years. Besides, who is to tell that the newer design will be better? Judging by the history of our industry, I have some serious doubts.

[pcwalton]

> Besides, who is to tell that the newer design will be better?

As much as it's popular to think otherwise, this isn't true. Taken as a whole, newer things are more often than not better than older things.

For example, cryptography has been on a huge march upward since the days of Unix crypt().

[rusk]

I have a shining counterexample to your thesis:

    https://arstechnica.com/apple/2015/01/why-dns-in-os-x-10-10-is-broken-and-what-you-can-do-to-fix-it/

In almost all my experiences reimplementing established software in a commercial environment "building a better wheel" has met with failure. Or in the very least it took some time to get it back to a level of acceptable that our customers were happy with.

This idea that it's easier to start from scratch than maintain old code is almost always borne of hubris.

The thing that many people miss is that software in active use is a "living thing" that has undergone many evolutionary iterations between developer and user before it's been gotten right.

There is so much implicit knowledge & experience embedded in the code itself. When you say you want to replace something with a "new" version you're effectively doing something similar to sacking an experienced engineer and replacing them with a graduate.

That's not to say that good enough is good enough. But it's important to remember that improving on "good enough" is hard. If you don't have a commercial mandate or other good reason for improving it it just may not be worth your time.

[markhahn]

Apple is in the vanguard of only one thing: marketing.

[rusk]

What a lame and supercilious comment.

[theoh]

Even the great Butler Lampson has a slide set online which includes the item

Design (Apple's forté)

This is arguably a slightly more defensible version of the GP.

[dredmorbius]

Cryptographic algorithms are, when they are shown to be better than old algos, better.

Many algorithms are shown to be not better than old algos, hence the frequent admonitions (see Schneier for multiple examples) against roll-your-own crypto. I'm trying to remember the name of a much-touted encrypted messaging application being advertised for use by Arab Spring activists which turned out to have massive vulnerabilities.

There's a list (which doesn't seem to include the one I'm thinking of) here: http://www.pcworld.com/article/2843372/popular-messaging-app...

The reality is that encryption advocates strongly encourage people to use tried and tested mechanisms.

Worse: cryptosystems, inclusive of the algorithm and all the support infrastructure around it are very frequently worse than old systems, and reveal very, very badly broken implementations, sometimes years down the road.

https://security.stackexchange.com/questions/18197/why-shoul...

ftp://ftp.pgpi.org/pub/pgp/7.0/docs/english/IntroToCrypto.pdf

[kbart]

Cryptography benefits mostly from huge computing resources we have today and fundamental advances (math). Current crypto algorithms were impossible/impractical with hardware from just few decades ago. Yet we still suck at designing crypto systems.

[rlucas]

In finance we call the driver of this "survivorship bias."

IOW, your contention is true/truthy if you ignore the fact that there is a large selective pressure weeding out the "not better" new things.

[TeMPOraL]

In computing industry, there's little pressure for weeding out "not better" new things. Unix is a great proof of that. Solutions survive on basis of popularity, not technical soundness, and many things get continuously forgotten and reinvented a decade later.

[JoeAltmaier]

Often for good reasons! In my career, I've seen the pendulum swing from storage on every device, to diskless devices sucking from the network, back to storage. Many times. Never mind the philosophy folks quote; its about technology, or rather the relative speed/latency of network versus local storage. Its true in lots of things. Folks 'rediscover' messaging systems for redundancy and failover every 10 years or so.

[pjc50]

Exactly. Backwards compatibility is king. There have been plenty of research operating systems which never made it to popularity; not just Plan 9 but things like BeOS and Nemesis. The network effects driving people to consolidate on one system are huge.

Browser design is a mess, but that's because it's a battlefield between competing demands. We're very lucky it's remained open and competitive. There's an alternate universe somewhere where IE6 won and you have to use ActiveX controls to enter your government ID before you can post to websites. (Oh wait, that's South Korea in this universe http://www.bbc.co.uk/news/technology-29617196)

You can write a better operating system, but unless it runs a browser and you can give it away for free it will have a very limited audience.

[arca_vorago]

As a sysadmin I just want to say I love bash. I'm not a programmer, but I get shit done in it all the time, and it's easy to understand and fix errors later. I have read every single why bash is bad article and I find all of them wanting for better arguments.

On the windows side, power shell.

And keep in mind, many of these scripts have been running for 10+ years. When's the last time you wrote a piece of software that worked for 10 years with minimal or no update?

[pif]

> and wonder if we can do better today.

No, we can't do better today. We may start something better today and enjoy it in 40 years, but for the time being, these are the tallest giants' shoulders we have.

[arpa]

Bash is not bad at all; it's unrestrained and arcane at times, agreed, but it does have the nature of a true programming language, just like powershell does. Now .bat scripts, that's truly inelegant and unsafe.

[ghettoimp]

While it's certainly not the worst, and certainly can be used for many things, bash is definitely pretty damn bad. You've got horrible string escaping nonsense ($IFS, ...), an utter lack of any sensible types, constant incompatibilities due to command-line tools supporting different arguments on different platforms, ...

While Perl or similar are not without many faults, at least they solve a lot of this. Think of even writing something like a JSON parser in bash... it's horrifying.

[arpa]

You could use something like jq for parsing JSON. And while I agree with you regarding some of your points, they could be easily refuted by pointing fingers at other things: yes, string escaping can give a grown man nightmares, but hey, it's not working with strings in C; and once you get the hang of it, it is pretty simple. Lack of typing can hurt, sure - but it's not BASIC; With a little imagination you can have strict types even.

And cli tools having different args on different platforms? come on, it's not even bash's fault...

Basically what i'm saying, everything you can do on $LANG, you can do in bash. And it's not that hard. It just has a loooong learning curve and seems to provide with just enough things to shoot yourself in the foot. But then again, you can be stupid in any language...

[ghettoimp]

I agree that CLI tools having different args on different platforms is certainly not bash's fault, but I think it hurts Bash much more than other languages because, since Bash is such a limited ("awful") programming language, the de-facto way of doing most anything in bash is to call out to these command-line tools for everything.

For instance, above you suggest using "jq" to parse JSON. I totally agree with you that if you actually needed to deal with some JSON data in Bash for some reason, then a tool like "jq" is definitely the way to do it, because it avoids trying to do actual programming in Bash.

In any "more reasonable" scripting language (perl/python/ruby/...), you can very easily open files, parse them, extract data, etc., all within the context of the language, without relying on all these external tools like awk/sed/grep/etc. And you have basic types like arrays and maps at your disposal that often make what you're trying to do a lot easier.

I can't imagine trying to actually write a JSON parser in pure bash. (Well, I can... but uggggh...)

[cat199]

> In other words, if you couldn't justify something being designed a certain way de novo, why be content with the existing design!?

Not justifying it designed this way today and keeping a false traditionalist position to retroactively-self justify it is no worse than berating something designed at the state-of-the-art at the time with no contextual understanding of why it was done that way... it is quite possible to evolve designs within a tradition of understanding their origins rather than being a smug neophyte who self-admittedly is responding out of ones own ignorance yet choosing to still berate despite this ignorance 'because who even knows why but it seems old man'

see also: just about any incremental improvement in any long running system ever, over time, across all systems, from the dawn of time, up until today

[walshemj]

Plain text config files are lot easier to write and edit than XML or JSON - the KISS principle applies.

[markhahn]

bash vs yml has always been there. you're really saying "declarative vs imperative" - not news. this is relevant, though, because Unix has always been about tools and composition, which is indeed dependent on imperative expression. if you feel dirty describing how (imperative), then maybe you're in the wrong business and should stick to the declarative world.