[ivan_gammel]

> So there's no need to continually patch it for security updates, or at least you need to do it less often, and it's less likely that you'll be dealing with breaking changes.

Regardless of how supposedly good or small is the library, the frequency at which you need to check for updates is the same. It doesn’t have anything to do with the perceived or original quality of the code. Every 3rd party library has at least the dependency on platform and platforms are big, they have vulnerabilities and introduce breaking changes. Then there’s a question of trust and consistency of your delivery process. You won’t adapt your routines based on specifics of every tiny piece of 3rd party code, so you probably check for updates regularly and for everything at once. Then their size is no longer an advantage.

> Copy-paste works, but I would argue the best way is to fork the libraries and add submodules to your project. Then if you want to pull a new version of the library, you can update the fork and review the changes.

This sounds “theoretical” and is not going to work at scale. You cannot seriously expect application level developers to understand low level details of every dependency they want to use. For a meaningful code review of merges they must be domain experts, otherwise effectiveness of such approach will be very low - they will inevitably have to trust the authors and just merge without going into details.

[ristos]

They don't need to understand the low level dependencies. People can create metapackages of a lot of a bunch of self-contained libraries that have been audited and forked, and devs can pull in the metapackages. The advantage is the modularity, which makes the code easier to audit and is more self-contained.

When's the last time ls, cat, date, tar, etc needed to be updated on your linux system? probably almost never. And composing them together always works. This set of linux tools, call it sbase, ubase, plan9 tools, etc, is one version of a metapackage. How often does a very large package need to be updated for bug fixes, security patches, or new versions?

[rrdharan]

> When's the last time ls, cat, date, tar, etc needed to be updated on your linux system? probably almost never.

Bad example: http://www.slackware.com/security/viewer.php?l=slackware-sec...

They find stuff like this fairly often in GNU coreutils even to this day.. it’s the main reason there’s a Rust coreutils effort.

[ristos]

It's probably still a good example. Looking up the CVEs for various search terms:

coreutils: 17 results

linux kernel: 6752 results

x11: 184 results

qt: 152 results

gtk: 68 results

docker: 340 results

rust: 455 results

python: 940 results

node: 110 results

javascript: 5657 results

firefox: 3268 results

chrome: 3763 results

safari: 1465 results

webkit: 1346 results

The large monolithic codebases have a lot more CVEs. I'd also argue that patching a fix on code made up of small, modular parts is much easier to do, and much lower hanging fruit for any casual developer to submit a PR for a fix.

[flysand7]

The large ~~monolithic~~ codebases have a lot more CVEs

Who would've guessed. Also the older ones also got more CVE's than newer ones, even if they aren't that big.

[layer8]

What you call an audited metapackage is nothing other than a non-micro-library. The property that it has been audited/assembled/designed/tested in conjunction and would be updated as a whole is exactly the benefit that non-macro-libraries provide.

[ristos]

It's not the same as a non-micro-library, because a non-micro-library is a bunch of internal code that isn't as well-documented and isn't self-contained, and maybe come and go as the non-micro-library continues to churn. I can't easily change a large monolithic library or project do better optimize for my use case. I could do that much easier though if the large thing was composed of a bunch of small self-contained things.

[layer8]

I was assuming that the constituents of the metapackage aren’t completely independent. TFA is about micro-libraries the size of is-number and left-pad. If you only consider completely independent libraries of that type, you won’t get very far. A more realistic take would be hundreds of such micro-libraries that aren’t self-contained, but instead build on each other to successively provide more powerful functions. And then the more transitive dependencies become more like “internal code” in regular libraries, and you’ll get the same churn, and you’ll be similarly unable to change things due to the interdependencies. I don’t see how you can end up with a comparable functionality a regular library provides without the same potential of running into the issues you note.

[ristos]

What if you're using a larger library and you wanted to swap out a sorting algorithm for one that's optimized for your use case?

I would say that the API boundary being more modular and explicit makes it possible to actually do those kinds of swaps if the larger library is composed of smaller modular code, in ways that you wouldn't be able to if it's buried in a bunch of internal code -- you would have to fork the library in that case.

[ivan_gammel]

> How often does a very large package need to be updated for bug fixes, security patches, or new versions?

I don’t think you understand my comment, because you are asking the wrong question again. It is not how often you need to actually update one dependency, but how often you need to check for updates that matters. That has to be done frequently no matter what and must be automated. E.g. despite low number of CVEs in coreutils you have to check them very often, because the impact can be very high. Once you have this process in place, there’s no advantage in using micro-libraries. I’d actually expect that in a micro-library environment most breaking changes happen when a library becomes unsupported and you need to choose an alternative, making the migration process more complicated than in case of a bigger framework which just changed the API.

[ristos]

Maybe I'm not understanding your argument. Are you saying that if all these larger programs wrote all those utilities from scratch, it makes it so that if someone messed something up, the rest of the large programs are unaffected?

All of those larger programs can mess things up in different ways, opening up all sorts of attack vectors and bugs. And a smaller utility that's much more easily auditable and easier to contribute fixes to is arguably less likely to have attack vectors in the first place.

I'm not sure you have to check large libraries less often. I would argue at least as much if not more often, because it's harder for people to audit larger codebases and to also contribute fixes to them. A significantly smaller number of devs can (or could, if they had bandwidth/time) understand a large self-contained codebase than a very small self-contained codebase.

I think that if a larger library is made of many smaller, modular, self-contained parts, that are composed together, and you can swap out any smaller building block for something that fits your use case (inversion of control), then that's a really good way to write a large library. Sloppier code might find it's way in due to time/resource constraints though, or the authors might not notice that some of the code isn't entirely modular/composable/swappable.

[ivan_gammel]

> I would argue at least as much if not more often

The same. The frequency of checks is not dependent on the library size and depends on the risk profile of your application, so library size cannot be considered an advantage with regards to updates. In most cases upgrade to newer version can be done automatically, because it is unrealistic expectation that developers will review the code of every dependency and understand it. Breaking changes likely occur with the same frequency (rare), albeit for different reasons, and impact can be on the same scale for tiny and large depenencies.