[silverliver]

This is the right answer. I'm willing to stick my head out and assert that languages with a "minimal" standard library are defective by design. The argument of APIs being stuck is mood with approaches like Rust's epocs or "strict mode".

Standard libraries should include everything needed to interact with modern systems. This means HTTP parsing, HTTP requests, and JSON parsing. Some laguages are excellent (like python), while some are half way there (like go), and some are just broken (Rust).

External libraries are for niche or specialized functionality. External libraries are not for functionality that is used by most modern software. To put your head in the ground and insist otherwise is madness and will lead to ridiculous outcomes like this.

[Arcterus]

> Standard libraries should include everything needed to interact with modern systems.

This is great when the stdlib is well-designed and kept current when new standards and so on become available, but often "batteries included" approaches fail to cover all needs adequately, are slow to adopt new standards or introduce poorly designed modules that then cannot be easily changed, and/or fail to keep up-to-date with the evolution of the language.

I think the best approach is to have a stdlib of a size that can be adequately maintained/improved, then bless a number of externally developed libraries (maybe even making them available in some official "community" module or something with weaker stability guarantees than the stdlib).

I find it a bit funny that you specifically say HTTP handling and JSON are the elements required when that's only a small subset of things needed for modern systems. For instance, cryptography is something that's frequently required, and built-in modules for it often suck and are just ignored in favor of external libraries.

EDIT: actually, I think my biggest issue with what you've said is that you're comparing Python, Go, and Rust. These languages all have vastly different design considerations. In a language like Python, you basically want to be able to just bash together some code quickly that can get things working. While I might dislike it, a "batteries included" approach makes sense here. Go is somewhat similar since it's designed to take someone from no knowledge of the language to productive quickly. Including a lot in the stdlib makes sense here since it's easier to find stuff that way. While Rust can be used like Python and Go, that's not really its main purpose. It's really meant as an alternative to C++ and the various niches C/C++ have dominated for years. In a language like that, where performance is often key, I'd rather have a higher quality external library than just something shoved into the stdlib.

[JoBrad]

The tradeoff of “batteries included” vs not is real: Python developers famously reach for community libraries like requests right away to avoid using the built-in tooling.

[Natfan]

I wasn't even aware there _was_ built-in tooling...

[bigstrat2003]

And yet, there are times where all I've had access to was the stdlib. I was damn glad for urllib2 at those times. It's worth it to have a batteries included stdlib, even if parts of it don't wind up being the most commonly used by the community.

[yawaramin]

The fact that there is a 'urllib2' implies that there's a 'urllib', which tells us something pretty important about the dangers of kitchen-sink standard libraries.

[wolvesechoes]

But nothing prevents a language to have rich and OPTIONAL stdlib, so that devs can choose different solutions without linking bunch of junk they do not use.

Really, good stdlib still allows you to use better suited 3rd party libraries. Lack of good stdlib doesn't add anything.

[auraham]

Related: Rust Dependencies Scare Me [1]

[1] https://vincents.dev/blog/rust-dependencies-scare-me/

[goku12]

> This is the right answer. I'm willing to stick my head out and assert that languages with a "minimal" standard library are defective by design.

> Standard libraries should include everything needed to interact with modern systems. This means HTTP parsing, HTTP requests, and JSON parsing.

There is another way. Why not make the standard library itself pluggable? Rust has a standard library and a core library. The standard library is optional, especially for bare-metal targets.

Make the core library as light as possible, with just enough functionality to implement other libraries, including the interfaces/shims for absolutely necessary modules like allocators and basic data structures like vectors, hashmaps, etc. Then move all other stuff into the standard library. The official standard library can be minimal like the Rust standard library is now. However, we should be able to replace the official standard library with a 3rd party standard library of choice. (What I mean by standard library here is the 'base library', not the official library.) Third party standard library can be as light or as comprehensive as you might want. That also will make auditing the default codebase possible.

I don't know how realistic this is, but something similar is already there in Rust. While Rust has language features that support async programming, the actual implementation is in an external runtime like Tokio or smol. The clever bit here is that the other third party async libraries don't enforce or restrict your choice of the async runtime. The application developer can still choose whatever async runtime they want. Similarly, the 3rd party standard library must not restrict the choice of standard libraries. That means adding some interfaces in the core, as mentioned earlier.

[qcnguy]

This is the philosophy used by the Java world. Big parts of the standard library are plugin-based. For example, database access (JDBC), filesystem access (NIO), cryptography (JCA). The standard library defines the interfaces and sometimes provides a default implementation, but it can be extended or replaced.

It works well, but the downside of that approach is people complaining about how abstract things are.

[goku12]

That makes sense. Just adding a clarification here. I wasn't suggesting to replace the standard library with interfaces (traits in this case). I was saying that the core library/runtime should have the interfaces for the standard library to implement some bare minimum functionalities like the allocators. Their use is more or less transparent to the application and 3rd party library developers.

Meanwhile, the public API of the selected standard library need not be abstract at all. Let's say that the bare minimum functionality expected from a 3rd party standard library is the same as the official standard library. They can just reimplement the official standard library at the minimum.

[CaptainOfCoit]

> External libraries are not for functionality that is used by most modern software.

Where do you draw the line though? It seems like you mostly spend your time writing HTTP servers reading/writing JSON, but is that what everyone else also spends their time doing? You'll end up with a standard library weighing GBs, just because "most developers write HTTP servers", which doesn't sound like a better solution.

I'm willing to stick my head the other way, and say I think the languages today are too large. Instead, they should have a smaller core, and the language designed in a way that you can extend the language via libraries. Basically more languages should be inspired by Lisps and everything should be a library.

[groby_b]

> everything should be a library.

That's exactly npm's problem, though. What everybody is avoiding to say is that you need a concept of "trusted vendors". And, for the "OSS accelerates me" business crowd, that means paying for the stuff you use.

But who would want that when you're busy chasing "market fit".

[CaptainOfCoit]

> That's exactly npm's problem, though.

I don't think that's the problem with npm. The problem with npm is that no packages are signed, at all, so it ends up trivial for hackers to push new package versions, which they obviously shouldn't be able to do.

[carols10cents]

Since Shai-Hulud scanned maintainers' computers, if the signing key was stored there too (without a password), couldn't the attackers have published signed packages?

That is, how does signing prevent publishing of malware, exactly?

[CaptainOfCoit]

> if the signing key was stored there too (without a password), couldn't the attackers have published signed packages?

Yeah, of course. Also if they hosted their private key for the signature on their public blog, anyone could use it for publishing.

But for the sake of the argument, why don't we assume people are correctly using the thing we're talking about?

[lelanthran]

In past comments I said that a quick win would be to lean on certificates; those can't easily be forged once a certificate is accepted.

[yawaramin]

How did Shai-Hulud get access to maintainers' computers?

[smaudet]

I don't think things being libraries (modular) is at odds with a standard library.

If you have a well vetted base library, that is frequently reviewed, under goes regular security and quality checks, then you should be minimally concerned about the quality of code that goes on top.

In a well designed language, you can still export just what you need, or even replace parts of that standard library if you so choose.

This approach even handles your question: as use cases become more common, an active, invested* community (either paying or actively contributing) can add and vet modules, or remove old ones that no longer serve an active purpose.

But as soon as you find yourself "downloading the web" to get stuff done, something has probably gone horribly wrong.

[TylerE]

IMO Python 2 was rhetorical gold standard for getting the std lib right. Mostly batteries included, but not going totally insane with it.

[999900000999]

It's not an easy problem to solve.

Doing it the right way would create friction, developers might need to actually understand what the code is doing rather than pulling in random libraries.

Try explaining to your CTO that development will slow down to verify the entire dependency chain.

I'm more thinking C# or Java. If Microsoft or Oracle is providing a library you can hope it's safe.

You *could* have a development ecosystem called Safe C# which only comes with vetted libraries and doesn't allow anything else.

I'm sure other solutions already exist though.

[pjmlp]

Why?

This is a standard practice in most places I have worked, CI/CD only allowed to use internal repos, and libraries are only added after clearance.

[brazzy]

Except that "clearance" invariably consists of bureaucratic rubber stamping and actually decreases security by making it harder and slower to fix newly discovered vulnerabilities.

[pjmlp]

Depends on the skills of the respective DevOps security team.

There are also tools that break CI/CD based on CVE reports from existing dependencies.

[Shorel]

> Doing it the right way would create friction, developers might need to actually understand what the code is doing rather than pulling in random libraries.

Then let's add friction. Developers understanding code is what they should be doing.

CTOs understand the high cost of ransomware and disruption of service.

[jajko]

Java is around for much longer, has exactly same architecture re transitive dependencies, yet doesn't suffer from weekly attacks like these that affect half of the world. Not technically impossible, yet not happening (at least not at this scale).

If you want an actual solution, look for differences. If you somehow end up figuring out its about type of people using those, then there is no easy technical solution.

[Ygg2]

> Standard libraries should include everything needed to interact with modern systems.

So, databases? Which then begs the question, which - Postgres, MySQL, SQLite, MS SQL, etc.? And some NoSQL, because modern systems might need it.

That basically means you need to pull in everything and the kitchen sink. And freeze it in time (because of backwards compatibility). HTML, HTTP parsing, and SHA1024 are perfectly reasonable now; wait two decades, and they might be as antiquated as XML.

So what your language designers end up, is having to work on XML parsing, HTTP, JSON libraries rather than designing a language.

If JS way is madness, having everything available is another form of madness.

[ivan_gammel]

It is not madness. Java is a good example of rich and modular standard library. Some components of it are eventually deprecated and removed (e.g. Applets) and this process takes long enough. Its standard library does include good crypto and http client, database abstraction API (JDBC) which is implemented by database drivers etc.

[Ygg2]

Yeah, and Java was always corporately funded, and to my knowledge no one really used neither the http client nor the XML parser. You basically have a collection of dead weight libs, that people have to begrudgingly maintain.

Granted some (JDBC) more useful than the others. Although JDBC is more of an API and less of a library.

[ivan_gammel]

HttpClient is relatively new and getting HTTP/3 support next spring, so it’s certainly not falling into the dead weight category. You are probably confusing it with an older version from Java 1.1/1.4.

As for XML, JAXP was a common way to deal with it. Yes, there’s Xstream etc, but it doesn’t mean any of standard XML APIs are obsolete.

[koakuma-chan]

My favourite is java.awt.Robot

[pjmlp]

Spot on, I rather have a Python, Java,.NET,.. standard library, that may have a few warts, but works everywhere there is full compliant implementation, than playing lego, with libraries that might not even support all platforms, and be more easily open to such attacks.

Is java.util.logging.Logger not that great?

Sure, yet everyone that used it had a good night rest when Log4J exploit came to be.

[ivan_gammel]

slf4j is probably more common now than standard Logger, and it was a good night for those who used Logback as implementation.