[ristos]

They ship together because all of those small composable units, that were once developed by random people, were turned into a meta-package at some point. I agree with you that randomly downloading a bunch of disjointed things without auditing and forking it isn't good practice.

I'm also not arguing against a large popular project with a lot of contributors if it's made up of a lot of small, modular, self-contained code that's composed together and customizable. All the smaller tools will probably work seamlessly together. I think UNIX still operates under this sort of model (the BSDs).

There's a lot of code duplication and bad code out there, and way too much software that you can't really modify easily or customize very well for your use case because it becomes an afterthought. Even if you did learn a larger codebase, if it's not made up of smaller modular parts, then whatever you modify has a significantly higher chance of not working once the library gets updated, because it's not modular, and you updated internal code, and the library authors aren't going to worry about breaking changes for someone who's maintaining a fork of their library that changes internal code.

[porcoda]

> all of those small composable units, that were once developed by random people, were turned into a meta-package at some point

No they weren’t. Every UNIX I used in the 80s and 90s shipped with those little composable building blocks as part of the OS, and GNU bundled them in things like coreutils forever. It’s not like there was some past time when there were independent little things like cat and wc and so on written by random people on the internet that somehow got bundled into a meta-package after they existed. That didn’t happen.

[ristos]

They were developed by different authors...

[inkyoto]

They have evolved together, though, not in isolation from each other.

[unscaled]

So are different functions and modules in the standard library of a large batteries-included language like Python, Java or Go.

But in fact, most of the traditional utilities you know and love were first implemented by a small team in Bell Labs, with the most "core" of what we now call coreutils (i.e. "ls") being written single-handedly by Ken Thompson and Dennis Ritchie. The other significant chunk of utilities (commands like vi, more, less, netstat, head) were developed by a group of people at UC Berkeley and were released later as the Berkeley Standard Distribution (BSD).

GNU Coreutils, as we known them, are mostly re-implementations of the original Unix commands (incidentally, a lot of them have been initially implemented by two guys as well - Richard Stallman and David Mackenzie). This is no small feat, but the GNU coreutils authors took good care to maintain compatibility with existing UNIX tools (and later with the POSIX standard when it was released). They didn't just randomly implement commands in the vacuum and waited for someone to come up and bundle them together. It's worth noting that if you're using a Mac, you're not even using GNU coreutils. Most of the core commands in macOS are derived from FreeBSD which traces its lineage back to the original UNIX implementation.

The fact is, most of the individual commands that are now in coreutils were never released individually by themselves - they were generally released as a distribution and usually developed together and specifically to interact with other tools. The Unix Philosophy could not have been developed if Unix did not have a basic decent set of core utilities to begin with, and in fact the core utilities like cat predate Unix pipe support (only introduced in version 3).

The availability of a reliable core system commands which are guaranteed to be available and follow a strict IEEC/ISO standard(!) was pretty important for the development of an ecosystem of non-core commands that are built on top of them. Imagine what would have happened if some commands used fd 0 for stdin and fd 1 for stdout, but others used a different file descriptor or perhaps even an entirely different output mechanism, such as using a system call or sending data through a socket. Interoperability would be much harder.

But this is exactly the case in JavaScript, especially within the NPM ecosystem. The lack of a proper standard library means that every developer has to cobble together the most minuscule of utilities that are usually available as part of the standard library in other ecosystems. But what's worse is that all the various libraries don't play very well with each other. You don't have enough standard base types that you can pass between the different libraries.

For instance, there are no reliable types for describing time (besides a crappy built-in date), input/output streams (until recently), HTTP client, sockets, URLs, IP addresses, random number generators, filesystem APIs, etc. Some of this stuff (like Fetch API and Subtle Crypto) exist in Node.js and have been locked behind feature flags since forever, but that effectively means that you cannot use thme. In the earlier days of the JS ecosystems things were much worse since we didn't even have a standard way for specifying byte arrays (Uint8Array) or even Promises.

[strawhatguy]

Who often worked together, sometimes, gasp in the same building! Definitely not random people on the internet not met or even emailed.

[ristos]

Then only use the software that Bob wrote in the same building you work at, I don't know what to tell you.

I don't think you can necessarily trust Bob's code without looking at what he wrote, just because you see him in person.

And coreutils has 200 contributors, many random people and strangers, from all over the world now.