[dn3500]

Separate error output was around for at least a decade before this. I know MTS had it in the 1960s, and I don't think it was their original idea. I used a CDC for a while and they had it too. So while this is the story of how standard error was introduced into Unix, it is not the origin story of the concept of standard error.

[lapsed_lisper]

Unix's standard error is definitely not the first invention of a sink for errors. According to Doug McIlroy, Unix got standard error in its 6th Edition, released in May 1975 (http://www.cs.dartmouth.edu/~doug/reader.pdf). 5th Edition was released in June, 1974, so it's reasonable to suppose Unix's standard error was developed during that 11 month interval. By that time, Multics already had a dedicated error stream, called error_output (see https://multicians.org/mtbs/mtb763.html, dated October 1973).

All the same, I'd be willing to believe that Unix's standard error could have been an "independent rediscovery" of one feature made highly desirable by other features (redirection and pipes). It's not clear how much communication there was among distinct OS researcher groups back then, so even if other systems had an analogue, Bell Labs people might not have been aware of it.

[dbcurtis]

The story that I recall about the origins of stderr is that without it, pipes are a mess. Keeping stdout to just the text that you want to pipe between tools and diverting all “noise” elsewhere is what makes pipes useable.

[kragen]

the article is about specifically what kind of mess and what kind of usability problems inspired the change

[temporarely]

I've always felt stderr should have been stdmeta.

p.s.

Well, actually more completely, something like this:

                  +---------+
    [meta-in] --> |         | --> meta-out
                  | p r o c | 
     input    ==> |         | ==> output
                  +---------+

[dotancohen]

You might like this proposal:

https://unix.stackexchange.com/questions/197809/propose-addi...

The idea is that some output is metadata (such as ps headers) and some is data. With stdmeta we could differentiate between the two.

[nerdponx]

Powershell has seven "output streams": https://learn.microsoft.com/en-us/powershell/module/microsof...

[MiguelX413]

That's awesome, I wish it was more commonly the case.

[jasonjayr]

I bet you'd also be onboard with files having data forks and resource forks too ... ?

TBH, it's a great idea, but history proved that we apparently prefer a single stream of data and solving all the problems it brings ...

[dotancohen]

We don't have a single stream - that's the point. stdout and stderr are already different streams.

[jasonjayr]

Right, I was alluding to the original Mac's filesystem, with separate data + resource forks, requiring all sorts of hacks to transfer files to and from them. Due to all the trouble of working with that across other platforms, Mac eventually gave that up at a filesystem level, and sprinkled ".DS_Store" files everywhere.

TCP/IP streams are bidirectional, but there is a limited way of sending "out of band" data, though it is not used as much. It would have been nice if the stdout/stderr multiple streams extended to TCP/IP networking and even HTTP messages too.

[LegionMammal978]

> TCP/IP streams are bidirectional, but there is a limited way of sending "out of band" data, though it is not used as much.

It's not real "out of band" data: that's something wholly invented by the Unix socket API. TCP itself just has an "urgent pointer", which addresses some byte further in the data stream that the receiver doesn't have yet, with the intent that higher-level protocols could use it as a signal to flush any data up to that pointer to observe whatever the urgent message is. There's nothing in the protocol itself to actually send a message separately from the rest of the stream.

[lapsed_lisper]

Among conceptually Unix-like OSes, at least one tried to do something along these lines: see http://www.bitsavers.org/pdf/apollo/SR10/011021-A00_Using_Yo... PDF pp 151 and following.

[temporarely]

Thanks. (page 151 for those interested)

[dfee]

This is interesting: MIMO channels to a process. Single stdin/stderr/stdout is effective for a single OS process, but with so much pulled up to user land (e.g. workers via green threads) maybe it makes sense to introduce multichannel i/e/o.

[euroderf]

Sure. Make metadata out-of-band rather than in-band, so that the ungovernable mess of Unix-standard plain ol' text streams is replaced by structured data.

So, well then: allowing programs to consume and emit JSON - is this progress ?

[leoc]

JSON is hardly the greatest structured format, but nearly anything is better than Unix text streams.

[somat]

I think having a plain byte stream is actually better, in a worse is better sort of scenario.

A plain byte stream can be easily aligned to work with any future or past encoding fashion. Consider the situation if them that designed unix had not been so aggressively minimal. We would probably be complaining how streams had to be ASN1 encoded and how much a pain it is to define the schema for what should be a simple ad-hoc data transfer.

As it stands, you can put whatever object format you want on top of the stream. I think it is the same with the files. I am sort of pleased we are not stuck with some obsolete no longer relevant, screwball structured format from the 70's that all our file have to conform to. instead our file are a simple range of bytes and we can impose whatever structure on them that we want.

[euroderf]

I'm thinking that if a range of XML markups can delimit and separate out metadata (e.g. HTML head v body), then heck so can JSON. Maybe not prettily.

[ElectricalUnion]

To me the advantage of XML in this would be CDATA segments, that can represent (almost) any text (that isn't the CDATA terminator string) without requiring text escape hell. And if some sort of JSON supported said CDATA, well, it is no longer JSON.

[BoingBoomTschak]

Why not more than 4? Another thing CL did "better":

https://www.lispworks.com/documentation/lw50/CLHS/Body/v_deb...

https://www.lispworks.com/documentation/lw50/CLHS/Body/v_ter...

[temporarely]

Sure, it is arguably cleaner to explicitly isolate error condition from general process meta-data. So in my OP diagram you can add an errout coming down from the proc-box. I've used this actual pattern in-process to hook up pipelines of active/passive components. However, there is no sense (imo) to propagate the errout of P(n) to P(n+1), so 2-in, 3-out.

p.s. That is pL(n).stderr -> pE.stdin, where pL is the 'business logic' and pE is the system's error processing aspects. I.e. the error processing component's stdin is the stderr of the logical processes (Lp), so there is a uniform process model applicable to both logical and error processing elements of the pipeline.

The issue is how to do this within the limits of line terminal interface (CLI). In code (as in in-process chaining) that aspect is a non-issue.

[pmarreck]

so... named pipes/named file descriptors?

[dredmorbius]

Similarly, the SAS System, originally written on an IBM mainframe in 1971 (probably OS-360 / MVS), and featured an input file (the SAS program itself) and two outputs, a LIST (the desired analytic output) and LOG, which contained status, warning, and error messages. It's not quite stderr, but clearly reflects similar thinking and was probably based on extant practices at the time.