[eloary]

I believe the right way to go is to focus on library code. If the long lived code is to prove exceptional it has to do so by being grabbed and bolted on to the new thing over and over, and that tends to favor an approach of libraries that assume very little, don't have many dependencies themselves, and opt for simple/robust API over being efficient. The API user can always recode the API for efficiency in their use case while as a library vendor your ability to guess at hotspots is limited at best, and as a consumer one is always looking for a library that can be used easily and disposed of quickly. In effect, "design by placeholder".

[eight_ender]

I don't argue your point here, but I'm seeing more and more developers that cut their teeth in the age of libraries that are terrified to touch library code.

There's an assumption out there for some that library code is flawless, and there's an imposter syndrome type aversion to touching it, with the developer fearing the code was made in a certain way for a certain reason and that they're not skilled enough to work on it.

This is made worse by the few brave developers that will dive in getting all the anger if they make a mistake, at a massive scale if the library is popular.

[yumaikas]

I suspect that the MS and Java tech stacks might factor into this a bit.

I can't speak for other languages, but in Go, the ability to navigate to the definition of any documented symbol via GoDoc has been most interesting for learning about how the sausage is made in certain areas. I don't have a comprehensive knowledge of all of Go yet, but I've learned that environment variables, for example, are backed by a `map[string]string`.

Perhaps something like that might help with the imposter syndrome about it?

[maccard]

I work on a closed source project using closed source libraries with some restricted source access. In general, if I find a bug in that library, it's less hassle for me to report it to them, and move on, than report it, fix it, and deal with the upgrade path later. It's unfortunate, but if I fixed every bug I found in the third party libraries, I'd never get any of my own work done..

[akkartik]

This tends to be mainstream opinion today: figure out the timeless essence at different scales, decompose it into abstractions, freeze their interfaces so that people can start relying on them, and so on. But it doesn't seem to have helped for forty years of trying to do it. The world changes too quickly, we aren't quite as good at designing libraries as we think, the world is filled with historical accidents in interfaces. With hindsight, it seems clear they were prematurely frozen.

Rather than dismiss these observations as isolated cases of people not practicing good behaviors, I tend to see them as evidence that we should be creating libraries far more conservatively, freezing interfaces far more late in the life cycle, perhaps even decades late.

If I'm right, we are also overusing industrial notions of assembly lines and division of labor. Libraries with non-trivial functionality take a long time to get right, and in the meantime they are produced more like guilds of craftsmen than factories. (Even if the products themselves permit factory-like operation at scale.) In that initial bake-in period we are ill-served by conventional metaphors of software components, building blocks, etc. We should be dealing more in vertically-integrated self-contained systems rather than plug-and-play libraries. More OpenBSD, less `gem install`.

I've been thinking about this for at least five years, ever since http://akkartik.name/post/libraries. More: http://akkartik.name/prose

[lmm]

Libraries do take a while to get right, but I think we can only get them right by making them as libraries. All good libraries I've seen started out as part of a vertically-integrated piece of software, but if there's currently an immature library for x, using that immature library generally puts us further along the path to getting a mature library for x than writing your own vertically-integrated implementation of x.

I'd focus on making it easier to migrate between libraries, easier to improve interfaces, and so on. (I'd argue to a certain extent that's already happened, and that's part of why we're using more and smaller libraries).

[akkartik]

Getting more users for x certainly helps mature it. However, making it a library also tends to freeze it. So ideally we'd have ways to encourage people to use something without guaranteeing its interface. I think this is a social problem; we need more libraries that have signs on them saying, "alpha software, compatibility not guaranteed," and we need greater awareness among developers that this is a good thing, that being willing to switch interfaces every once in a while results in a better eco-system in the long term.

[lmm]

Agreed. But I think the idea of listing and cutting down your dependencies (which is what I understood your post to be suggesting) is contrary to that; rather depending on a huge number of tiny libraries makes it easier for libraries to evolve more flexibly.

[akkartik]

In general I don't see the connection between the size or number of dependencies and how easy they are to evolve. Truly tiny libraries like left-pad have trivial implementations. Why not just inline them into your project? Then they're no longer libraries, just functions. You get the abstraction benefits, but you don't need to enter into a counter-party relationship with the author.

At scales above the absolutely trivial I think libraries evolve more flexibly based on social rather than technical considerations. It's about not pissing people off when you change the interface. Even something as complex as Go was able to make incompatible changes pre-1.0.

[lmm]

My sense is that smaller libraries are more able to evolve at their own place - e.g. the separation of django-rest from django core, or languages moving more things out of their standard library. Pre-1.0 is the easy part, we'll see how Go does is 5 or 10 years once it has a base of programs to maintain compatibility with - I predict it will find it harder to evolve and its library will fall behind.

[billysielu]

I agree. It's especially pungent in JavaScript, in which the average 'JavaScript developer' is playing some perverse game of library pokemon.

[graphitezepp]

Definitely stealing Javascript Pokemon. Seriously why are people still adopting more libraries to perpetuate this insanity.

[_xkjr]

Further thought: this is a similar problem to creating too many standards. https://xkcd.com/927/