Hacker News new | ask | show | jobs
by steveklabnik 2078 days ago
https://wiki.alopex.li/LetsBeRealAboutDependencies

> These complaints are valid, but my argument is that they’re also not NEW, and they’re certainly not unique to Rust ... The only thing new about it is that programmers are exposed to more of the costs of it up-front.

4 comments

Hm, this response essentially says "other languages have this problem too, so deal with it".

While thats true, it completely misses the point. The point is not that dependencies exist, or even that a package might have many dependencies.

The point is, Rust (and NPM) I have found many times dont care or even consider the impact of a large amount of dependencies, and often take no steps to mitigate or reduce that number.

As others said, some features could be split off into other crates. Maybe someone only needs HTTP, or maybe they need HTTPS but no Async. Or maybe they dont need logging. With Hyper and others you just have to build everything whether you want it or not.

> As others said, some features could be split off into other crates.

I agree with this, but that's not what your original post says. Or at least, it’s not what I understood from reading it. :)

> The point is, Rust (and NPM)

and C, in many real-world cases, which is why the above post matters.

When I find a project that is a handful of .c files and a Makefile they almost always compile and run. Sometimes with warnings because the features used in the code are depreciated but usually without too much fanfare.
> they almost always compile and run

Same in Rust. Actually, it's quite better than in C. The only time Rust projects fail to compile is when they pull in some C library and something there (like configure.ac) messed up. :D

And if this C project does anything interesting it pulls in bunch of C libraries that came precompiled with your OS, and might be stale and contain unpatched security vulns.

C/C++ developers pointing at other languages about dependency hell is a curiosity.

> other languages have this problem too, so deal with it

This is the primary reason I try to avoid projects built with npm. Fucking dependency hell. If the project hasn't been actively maintained in the last 3 months your chances of getting it to work drop precipitously.

Github and npm are both graveyards filled with dead JS libraries. They make it too easy to litter the universe with sub-par orphaned software. And somehow, it's up to each individual to filter it all out. You have critical software such as React sitting next to mountains of bad nonsense code. And they are all on equal footing.

People love to trash Perl on HN. But among many other things that Perl devs understood, they deeply understood issues that come up with dependencies. Most CPAN modules are namespaced, have unit tests, and unit tests run when modules are installed. Not only that, the people behind CPAN understood that it is a community effort and you, as a library author, have certain responsibilities to your community.

https://pause.perl.org/pause/query?ACTION=pause_04about

None of that exists in npm. We have scopes in npm, and that's about it. CPAN makes npm look like a child's toy.

Can you point out any projects that have gone off the rails because of dependencies? I mean otherwise it's just "what if" syndrome.
> The only thing new about it is that programmers are exposed to more of the costs of it up-front.

That's funny, because it's only "new" if your experiences primarily lie in newer languages and communities. There's a lot of criticism of C, but one thing it does is make dependencies pretty explicit. Some say that's good, some say that's bad, I guess it can be both at different times.

I don't believe that's accurate in general.

Let's say you open up a new C codebase: What are its dependencies? You'll have to hunt through its README (hopefully it's up to date!), other build instructions, maybe CMake, maybe some custom build system, etc.

What version of dependencies does it use? If the code has been vendored, you at least know what code its using - but where do you look for updates to that code? Do you manually go out to wherever it was copied from now and then and look for updates? If the code isn't vendored, then how was it installed? From the package manager? If so, what operating system and version was used during development? If its not from a package, its might have been downloaded and installed manually? Again, where was it downloaded from? Where was it installed? What options did it use when it was compiled?

How do you handle transitive dependencies? Probably by hand. How well documented are they?

C suffers plenty of dependency issues.

But each one of these dependencies is pretty consciously and manually added, most of the time. In new code, to introduce a new one usually requires thought, and that creates a culture of caution.

Also if you are dynamic linking, ldd(1) can give you a pretty good picture.

And a simple ‘cargo tree‘ will show you the tree of dependencies for a crate, nicely formatted.
This. Take even CURL as an example, try to list its dependencies and you'll see how harder it is.
libc that's it. Every other dependency for curl is optional.....
Try it. If you do not use any "optional" dependency it becomes pretty limited, almost useless for anything serious (eg. zlib, ssl)
Zlib is pretty small.

TLS implementions are often giant hairballs but one that many things depend on. You can think of it as a somewhat "system level" dependency.

"It has always been that way." doesn't seem to be a helpful statement here.
I mean, in some sense you're right, but in another, the point is that it's not a regression.
True.

Also, I coud imagine Rust's type-system raises the bar of dependencies that can be mangaged before everything breaks down. So a lib with 100 deps in NPM isn't the same as a lib with 100 deps in Cargo.

If you lock to specific versions, I don't think it differs much.

Edit: I mean if you don't use ~ or ^ in your nose dependencies,. Just explicit versions.

Yes but people often throw out empty criticism without offering solutions or alternatives and it gets annoying after a while and is just a complaint rather than a healthy critique. As programmers we've heard them all before I guarantee it, so why add another one to the pile?
Saying "The people complaining are right, but they've been right a long time" isn't a great endorsement of the situation.
I mean, you're just repeating a sibling comment, but if development has been this way for a long time, it's on the folks who are suggesting the new way to get out there and prove that it's a viable model for software development. It appears that most real-world, actually used software works like this.

I am all about improving the world, don't get me wrong, but saying "hey this software works just like all the other software" isn't really the insult that you seem to think that it is.

> saying "hey this software works just like all the other software" isn't really the insult that you seem to think that it is.

Well, it's common knowledge that most existing software is compete and utter crap, as evidenced by the fact that our first thought upon hearing that a particular piece of software is no longer being updated is not "oh good, it is (probably) finished and we can rely on it", but rather "on no, now the innumerable defects no doubt still latent in it will remain unfixed". So "this software is just as bad as all the other software" is, while not a very grave insult in a relative sense, still quite damning in absolute terms.

It depends. If it's in a github repo and there isn't a massive backlog of issues for a software that hasn't been updated in a while, I might think that.

One good thing about stat counters for packages combined with GitHub for issue tracking of you can kind of tell.

It does take some level of die diligence and isn't easy. But neither is anything relying on say system installed libraries in C projects.

I'd rather have the package managers than not.

Can you give even a single example of:

- a significant (eg, at least as complex as wget) software project,

- that has been unmaintained (no updates, code has the same MD5/etc hash),

- with a significant userbase (not sure exactly how to define that one),

- for a significant amount of time (at least five years),

- which is generally regarded as finished and bug-free (not in need of further development) rather than abandoned?

Because I can't think of a single one, and the only ones that even come close are video games where the known bugs were co-opted into gameplay features. The general consensus seems to be that any system that doesn't have automatic updates running is de-facto insecure (which, since every update mechanism I've heard of can introduce new code (ie new security vulnerabilities), means any system whatsoever is insecure).

(I don't quite disagree with the tacit assertion that actually getting things right on - if not the first try - then at least one of the first thirty or so is a extremely, maybe even unreasonably high standard, but it manifestly is a standard that basically all existing nontrivial software projects fail to meet.)

5 years is a relatively rough one... in terms of libraries, I come across a lot that are 2+ years old that are feature complete and work. In terms of applications, there are a couple other responses in this thread, but the specific focus in reference was really on libraries themselves, which shouldn't be as complex as wget in general.
It seems to me Rust folks have developed this habit of deflecting blame by pointing to shallow commentary. One of good examples is compiler slowness reasons, sometimes its LLVM, or it is lot of optimizations, or it is not really slow compare to C++ and so on.

They could have said it straight "Guys highly optimized, safe compilation of medium size project will be in range of 20-30 min". And that would great and honest way to deal. Instead we get oh, we have reduced compile times from 26 minutes to 21 minutes so it is 19% improvement in just one year and there is more to come. Now this is hard work and great but I am sure Rust committers understand when people say fast compile times they most likely comparing to Go etc which would be under a minute for most mid size projects. And this is very well not going to happen.

Same is now to Cargo dependency situation. Cargo and NPM are ideologically in agreement that people in general should just pull it from package manager instead of rewriting even little bit of code. And again instead of owning it there will be list of shallow reasoning: others do it too, reusing is better that rewriting, cargo is so awesome that pulling crate is much easier and so on.

Your argument seems to be "people see problems and other people explain why those problems exist" and somehow you frame it like it's a bad thing.
No my point is Rust fans try to win very narrow technical arguments even when they should clearly know discussion is about big picture. And yes it seems bad thing to me.
On an internet forum when someone brings up a topic lots of people will respond with different opinions about that topic. You can act like this is somehow specific to the Rust community, but I don't think it is.

TBH you seem to have a really odd bias against Rust, you repeatedly take something about it that's positive and try to spin it as negative, like possibly even for years? Maybe examine that.

Rust core leadership treats most technical problems as marketing problems.