Hacker News new | ask | show | jobs
by malkia 28 days ago
Everytime I see a language creating their own package system, all I can think of it how much we've missed here.

The only exception is C/C++, where there is none established that well, for good or bad.

These choices may create later super-convoluted processes when you have to mix more than one language together.

Packaging systems makes thing easy, but complicate further the line if another language needs to be used.

6 comments

What do you think we've missed? Do you want one build systems for all languages? There are such systems (e.g. Bazel) and they're often used for multi-languages projects, but I think reality has proven that build systems with language-specific knowledge are much easier to navigate.
I'm not sure how outlandish we're allowed to get here, but IMO we have a fundamental mismatch between application <> operating system <> CPU in terms of dependencies and trust.

Fixing this is beyond any one tool, of course :)

Cross platform building and packaging in C/C++ is such a hot mess. There's so many dimensions to unpack I don't even really know where to start. (I say this as the person who has been packaging GIMP for Mac for the last good number of years.)

OK, here's a few (with a MacOS slant): - compilers (gcc, clang, and their many versions) - libc (and friends) compatibility (I can't say I even ever delved into this one, but it's bit me) - package manager (macports, homebrew) - building for backward compatibility; what's the earliest MacOS version to support (the package managers either like to build for the OS they are running on, or force you -- yes I'm looking at you homebrew) - dependency and dependency version management (love you pkgconfig) - build system for each package (cmake, autotools, meson, ...) - bundling everything into an application - turning that application into a Mac application - code signing and notarization - creating the DMG - debug symbols - crash detection and notification Like right now, libheif on 26 can't be built for 11 and it's not clear why (or maybe they just fixed it...but it's been weeks)

C should have fixed its various issues and added a package manager (or blessed one), Zig is filling that gap.
No it's not. Not everybody uses Zig. Rust/Python/And others have also tried "fixing" it.
I think it's actually good that C++ has no standardized packaging system. This forces one to think carefully before introducing a dependency, since often such dependency have hidden costs, like security vulnerabilities. Since many critical systems are written in C++, it's too much risk to depend on dozens of easily-accessible third-party packages without properly auditing each of them.
I'm sorry, I have to address this take every time someone brings it up. The lack of a modern, ubiquitous, cross-platform packaging system is an absolutely terrible thing for C++.

I've worked on many large projects in C++, and every single one of them contains a bespoke, buggy, undermaintained JSON parser, URL parser, configuration file parser, async framework, and so on. It used to be the case that almost every large C++ project started out by defining its own friggin' string type.

Dependency anxiety is a variant of NIH syndrome, and it leads to much, much worse quality software in the average case. Most companies are not in the business of writing a bug-free async framework, and yet here we are. The cost of vetting your dependencies is much, much lower than writing and maintaining all these things from scratch.

I didn't say one should not use thirdparty dependencies at all. They are sometimes useful. But they should be chosen carefully and ideally reviewed. And any updates should be done manually in order to prevent security chain attacks.

Having a standardized package manager allows lowering the bar and bypassing careful thinking. It has also a cumulative effect - if one adds each dependency in its project one by one with proper audits, transitive dependencies may not be managed so carefully. And then we have cargo-style cancer with trivial projects having hundreds of dependent packages.

The C++ style tends to create much larger omnibus libraries. If you're concerned about the liability and bloat extra dependencies create, you should be thinking of a) the number of people you are trusting in your supply chain, and b) the total amount of code you are importing. Neither of these correlate directly with the number of different packages that appear in your package manager, and in fact cargo-style splitting can allow you much more fine-grained control over what code appears in your application.

Probably the one security sin of most language package managers is allowing anyone to upload to the central repository without review. This is good for accessibility but bad for security. There are tools like cargo-crev, though, which can help you enforce some level of vetting if you wish.

I don't think masochism is a reliable or sound security strategy.
On the other hand, systems like cargo a clearly a supply chain disaster.

I also think that languages should not have their own specific packaging system. This should be done on a distribution level, which provides curated lists of software. This system works well in the Linux world. The problem is the support for the commercial platforms.

The whole trend to think of programming languages as closed frameworks is bad. It is replacing a better system with one which is fundamentally much worse, instead of trying to fix the problems with the existing systems (which is hard because it needs collaboration and harmonization, instead of going into your own little bubble and pretending to do something superior).

> On the other hand, systems like cargo a clearly a supply chain disaster.

I don't think that has been shown to be true. There's a lot of FUD, though. Tools like `cargo vet` and `cargo audit` seem to be pretty good.

> I also think that languages should not have their own specific packaging system. This should be done on a distribution level, which provides curated lists of software. This system works well in the Linux world. The problem is the support for the commercial platforms.

I mean, it works until it doesn't. There are really significant drawbacks to this as well, including the knack some distributors have for thinking they know better than the original developer.

If you use a JSON parser, URL parser, configuration file parser, async framework, and so on, maybe you have other problems. Not that I don't know these project or don't have to work on them, but I don't think it matters what JSON library or URL parser library or async framework you're using.

The actual problem is the complexity coming from the choice to use these things at all (and thinking they should play any important role in the system at all, instead of being entirely replaceable), which means to do anything useful you have to go through a lot of abstraction, which makes everything tedious and buggy and hard to fix.

Conan and vcpkg are getting there.

It is as standard as in many languages where package management came after years of using the language.

The world has yet to standardize on a good crossplatform polyglot build system.

The only real such build systems are Buck and Bazel. But they have way too much baggage from their overlords.

It’s a shame.

Zig is pretty good for my use case. It may not be fully pollyglot at a technical level, but I can use it for my embedded C use case, for Jank to export the .cpp to other platforms, and thereby Clojure.
build systems using llvm as the backend are getting there, but zig is making their own compiler too.
compilers and build systems are and should be different.

A good polyglot build system should support llvm and zig and python and literally any toolchain under the sun.

the problem is that you need a compiler capable of compiling said zig, python, rust. LLVM does that, you can compile something as untyped as javascript(with a lot of hacks) or as strict as rust.
> the problem is that you need a compiler capable of compiling said zig, python, rust

no! No you do not!!

It’s perfectly totally fine to have multiple different compilers. Literally not a problem at all. That’s my point!

Compilers and build systems are separate. It is perfectly fine for one build system to invoke multiple different compilers.

cargo pretty much does that, but then you have to have to bundle all those compilers which becomes a fuckfest.
Conan and vcpkg are established enough.