[TheDong]

Go included json in the standard library, so it can't ever change it.

It's wildly inefficient (mostly due to reflection), not type safe, and somewhat surprising. Guess the output https://go.dev/play/p/lSyiaN3IYOR

The author also mentioned "Images manipulation" as a thing that should exist in the standard library.

We actually know how that one went from go too: There's the stdlib image/draw (https://pkg.go.dev/image/draw). But generally, it's recommended to use the non-stdlib golang.org/x/image/draw package instead these days (https://pkg.go.dev/golang.org/x/image/draw)

For backwards compatibility reasons, the stdlib couldn't change, and it's actually far more confusing as a result.

I take a much different lesson than the author of this post about what lessons to take from Go's "large stdlib" experiment.

[KronisLV]

> I take a much different lesson than the author of this post about what lessons to take from Go's "large stdlib" experiment.

Personally, i think that this is simply a problem of not being able to throw the entire thing away, learn from the mistakes and make a better iteration.

I'm all for "batteries included" anything - be it a standard library for a programming language, an IDE with its own component libraries like Lazarus, a web development framework like Angular that's less fragmented than React or maybe even an operating system that has a lot of the software that you'll want to use preinstalled (or available on a DVD or a similar medium directly).

It's just that making sure that those batteries are good is impossible without iteration. Having to do backwards compatibility essentially makes that sort of iteration impossible. So what's the solution here?

Have a clearly defined end of life for the current version and a new major version in the future. Python 2 might have had a lot of warts, but Python 3 would clearly be better! One could say the same for Python 4, Python 5 and so on. The same goes for Go, Rust and any other high level language - just look at Java 8 and the newer iterations. You just need to learn to throw old things away instead of keeping them around in some half-alive state.

I say that as someone who's currently maintaining a Java 8 monolith that cannot be updated to anything more recent. If your software gets so large to the point where it cannot be migrated/rewritten in a manageable way, you've made a mistake. If you cannot feasibly migrate it over, then consider whether you even need it in the first place (e.g. build systems that you do less).

There are sub-optimal cases where that simply isn't possible: the Linux kernel comes to mind, as does a lot of other large legacy software that's slowly trudging along on the backs of tens of thousands of developer years. But somehow you don't see many new/existing systems using FORTRAN or COBOL and that's all for the better.

[TheDong]

The lesson that I take is that all non-essential packages should instead be in an official, separately versioned, stdlib.

Go moved "image/draw" into "golang.org/x/image/draw". This was a good solution because it meant you could update the compiler without having to worry about also updating your image drawing stuff and that possibly breaking.

I wish they did that with http, tls, and all the other stuff that isn't really part of the language itself.

As is, I usually have to wait months to get compiler benefits (like being able to use generics, or smaller binary sizes, or fixes to bugs) because the compiler update also comes with breaking changes to the http package (like the http2 change) or tls or whatever.

My lessons is that there should be 3 tiers of libraries:

1. Compiler-versioned libraries. Things like io, syscalls, reflection, and things who's implementation depends heavily on the compiler internals. Breaking changes are never made without extreme circumstances.

2. Official, separately versioned, standard libraries. Things like golang.org/x/ or the idea behind the rust-lang-nursery. This is for stuff like http, logging, tls, crypto algorithms, image manipulation, high-level filesystem APIs, other protocols and abstractions. It is supported to use an old version of these libraries with a new compiler forever, and the compiler will never change libraries in 1 in such a way that they could break any old version of 2. You should update these whenever you can, but they may make breaking changes, and may be versioned as such.

3. The rest of the packages, stuff the community writes. These should prefer to use semver, and should prefer to use libraries from 1 and 2, but no promises are made.

I think having these 3 tiers is ideal, especially because it makes it easier to update the compiler (it will never break you), ensures perpetual support for older versions of official non-stdlib-but-still-core libraries (may make major releases that break, but the old one builds and works forever).

This is already pretty close to how C works actually, and it's great. Tier 1 is the actual language spec itself, linking libraries, etc. Tier 2 is libc. Tier 3 is the rest of the world. I really don't have to worry about how my pthread_create call in glibc will break when I update gcc because that separation is well maintained, and backwards compatibility is assured.

Python and go are both a clear step back, where I update the compiler to get some feature, and then my http server stops working even though I had no intention of updating it.

[KronisLV]

> The lesson that I take is that all non-essential packages should instead be in an official, separately versioned, stdlib.

Hmm, honestly, this is a really great take. A thin OS base, with the "batteries" being optional. Or at least being to use specific versions, bits and pieces from various points in time as needed.

Of course, this could turn into an Eldritch mess, but then again, it might also lead to working software over not being able to update the stuff you want to update because the rest would break otherwise.

Overall, there's probably a lot of consideration to be made about the details, but that seems workable. Stability might still sometimes be an issue (e.g. should the language internals change, a la Python 2 to 3), though i'm not sure whether that can even be addressed all that well, at least in the higher abstraction level languages (with separate runtimes like CLR, JVM etc.).