[dietr1ch]

I don't think libraries are the problem, but we don't have a lot of visibility after we add a new dependency. You either take the time to look into it, or just add it and then forget about the problem (which is kind of the point of having small libraries).

It should be easy to build and deploy profiling-aware builds (PGO/BOLT) and to get good feedback around time/instructions spent per package, as well as a measure of the ratio of each library that's cold or thrown away at build time.

[taeric]

I agree that I don't like thinking of libraries as the problem. But they do seem to be the easiest area to point at for a lot of modern development hell. Is kind of crazy.

I'll note that it isn't just PGO/BOLT style optimizations. Largely, it is not that at all, oddly.

Instead, the problem is one of stability. In a "foundation that doesn't move and cause you to fall over" sense of the word. Consider if people made a house where every room had a different substructure under it. That, largely, seems to be the general approach we use to building software. The idea being that you can namespace a room away from other rooms and not have any care on what happens there.

This gets equally frustrating when our metrics for determining the safety of something largely discourages inaction on any dependencies. They have to add to it, or people think it is abandoned and not usable.

Note that this isn't unique to software, mind. Hardware can and does go through massive changes over the years. They have obvious limitations that slow down how rapidly they can change, of course.

[dietr1ch]

> Instead, the problem is one of stability. In a "foundation that doesn't move and cause you to fall over" sense of the word. Consider if people made a house where every room had a different substructure under it. That, largely, seems to be the general approach we use to building software. The idea being that you can namespace a room away from other rooms and not have any care on what happens there.

I'm not sure what the problem is here.

Are you after pinning dependencies to be sure they didn't change? Generally I want updating dependencies to fix bugs in them.

Are you after trusting them through code review or tests? I don't think there's shortcuts for this. You shouldn't trust a library, changing or not, because old bugs and new vulnerabilities make erring on both sides risky. On reviewing other's code, I think Rust helps a bit by being explicit and fencing unsafe code, but memory safety is not enough when a logic bug can ruin your business. You can't avoid testing if mistakes or crashes matter.

[taeric]

Stability in that you don't want to take on a dependency that will throw a migration at you within the next decade. Or longer. You also don't want one that will introduce enabled sweeping features in the common path.

Examples: Google's Guava for the migration department. Apache Commons would be a good example of how not to make life painful for users there.

For sweeping features, Log4j introduced some pretty terrible security concerns.

[dietr1ch]

> I'll note that it isn't just PGO/BOLT style optimizations. Largely, it is not that at all, oddly.

Well, it's not required to trim code that you can prove unreachable, true. But I was thinking about trying to measure if a given library really pulls it's non-zero weight, and how much CPU is spent in it.

A library taking "too much time" for something you think can be done faster might need replacement, or swapping for a simple implementation (say the library cares about edge cases you don't face or can avoid).

[taeric]

Fully agreed in that this can be heavily subjective. And some things are flat out difficult with no real way of saying what is "too big."

My point on PGO/BOLT not being relevant was more that I see people reaching for libraries to do things such as add retries to a system. I don't think it is a terrible idea, necessarily, but it can be bad when combined with larger "retries plus some other stuff" libraries.

Now, fully granted that it can also be bad when you have developers reimplementing complicated data structures left and right. There has to be some sort of tradeoff calculation. I don't know that we have fully nailed it down, yet.