[awesome_dude]

Package management is the bane of nearly every language/technology

Nobody has "solved" it, and I don't think that there will ever be one (never say never, though, right?)

For Go we rely on developers of libraries to adhere to the semver versioning scheme accurately, and we cannot "pin" versions (a personal bugbear of mine)

There is a couple of workarounds - using SHAs not unlike the git commit hash to provide a pseudo version, and, vendoring (which is a cache of known dependencies - which brings with it cache management problems)

I had the misfortune of having to use Python with a virtual env on the weekend - it did not end well, and reminded me why I migrated away from Python.

Look at Perl (cpan) Java (maven, gradle) Ruby (gems) Go (dep, glide, vgo, modules) Rust (cargo) Node (npm, yarn, etc)

OSes too Redhat (yum, rpm, etc) Debian (apt) Ubuntu (snap - god why????)

And so on

[mxey]

Actually with Go modules you are always pinning dependencies. What’s in your go.mod is what is used. If your go.mod needs to be updated because a dependency wants to bring in a newer version of a transient dependency, the go.mod has to be modified (by the go command, not by you)

[awesome_dude]

I don't think you understand the term "pinning"

go mod tidy will update your go modules whenever it feels it needs to and there's nothing you can do to stop it.

The workaround is vendoring, where you control the versions in a cache.

[kbolino]

There is something you can do to stop it actually. You can use a replace directive, specifying that a module is replaced by itself at a fixed version. See e.g. https://stackoverflow.com/a/77412524/814422

It is worth noting though that, even without such pinning, `go mod tidy` does not update versions willy-nilly. [edit: the following is inaccurate, see grandchild comment] It only syncs go.mod with what is already being used by the build process. In other words, if you see `go mod tidy` change a version, it means that you haven't tidied the file since making other changes to it, and the listing in go.mod was stale with respect to the resolved set of transitive dependencies actually being used.

[mxey]

> It only syncs go.mod with what is already being used by the build process

If dependencies are incomplete, Go will fail to compile and tell you to run go mod tidy to fix it.

[kbolino]

Indeed, I ran two tests (missing indirect dependency, stale indirect dependency version) and it refused to compile both. Either what I said was never true, or it was only true for earlier versions of the `go` command. Nevertheless, adjusted accordingly, I believe the following statement is true: `go mod tidy` doesn't change versions in go.mod unless it needs to, to satisfy the other dependencies listed in go.mod, or to fill in a missing dependency for an import in code. It would be nice if there were a flag to turn off the latter behavior, though.

[mxey]

Yes, initially modules used to modify the file automatically which is why I have -mod=readonly in some old pipelines.

I think the “new” way is much better.

I still run tidy in pipelines to check but that’s only for cleanup.

[mxey]

Pinning to me means there is a file with all the versions as they will be used. I don’t see how “go mod tidy” modifying it is different from “bundle install” modifying it.

[dxdm]

> I had the misfortune of having to use Python with a virtual env on the weekend - it did not end well, and reminded me why I migrated away from Python.

I see this sentiment a lot, and it doesn't match my experience at all.

In my decade-old bubble of using Python professionally, I've never had an issue with virtualenvs. The few issues I might've had with dependency resolution must be so far in the past that I don't remember. But that's not strictly about virtualenvs. Likewise, pip could be clunky, but we don't have to deal with it anymore.

My niche is mostly backend. Other Python niches must be considerably worse in this regard.

[awesome_dude]

I used Python for a decade (professionally), gave up on it once I started using Go (professionally) in earnest - about 8 or 9 years ago.

I never liked virtual envs, having to remember where they were, what their names were, and what was installed into each one was a pain point for me.

This weekend I was trying to learn some AWS stuffs, and I cloned the official repo of example code which was Python. I followed the directions exactly and ... boom Python versioning issues... inside the freaking venv

Who needs that?

Why do I need to spend the better part of a couple of hours debugging a versioning problem? (FTR The problem turned out to be the repo was hardcoded to 3.8 and my local Python was 3.9.. or something along those lines - you are welcome to correct me, but that's what I remember of a painful waste of my time)

With Go I have backward compatibility guarantees - usually (there have been instances in the past where the backward guarantee have been broken AND the build process got broken hard for modules, with the claim that it was external and therefore not subject to the same guarantees)

> I see this sentiment a lot, and it doesn't match my experience at all.

My old HCI professor used to tell me - if users are complaining (or producing workarounds like post-it notes on their monitors) - regardless of how clean or elegant you think the system is - it's not.

You're saying you see people complain about it a lot - therefore it's a genuine problem.

[andrewflnr]

> having to remember where they were, what their names were, and what was installed into each one was a pain point for me.

It's at the project root, it's named 'venv', and its contents are described by requirements.txt.

> You're saying you see people complain about it a lot - therefore it's a genuine problem.

Debatable as a principle, but applicable enough here I suppose. Still, I'm not saying the problems aren't real, but what I (and probably most of us virtualenv users) are saying is that there's a pretty broad swathe of projects where you don't encounter them. It's just fine. You install your packages and use your packages and that's the whole story.

I guess if you have a hard dependency on a particular version of python, it's going to be harder, but... why? That's already niche in my book. If you're saying the AWS repo was pinned to a particular version of python, I'm going to blame that on Amazon frankly. That's definitely bizarre.

Edit: Were you looking at this? https://github.com/boto/boto3 Definitely more complicated than a typical greenfield virtualenv-able project, with some python version restrictions.

[awesome_dude]

> what I (and probably most of us virtualenv users) are saying is that there's a pretty broad swathe of projects where you don't encounter them.

Zero. The required number of problems needs to be zero - hence my OP

> I guess if you have a hard dependency on a particular version of python, it's going to be harder, but... why?

The bigger question is - why doesn't 3.9 compile and run 3.8

Further, in what world is targeting a specific runtime version in an enterprise production environment "niche"?

When you are deploying to managed corporate infrastructure, AWS Lambda runtimes, or strict Docker base images, you don't just get to loosely target "whatever Python version happens to be on the developer's laptop." You target an exact runtime version (e.g., Python 3.10) because language syntax, standard library features, and performance characteristics change between minor releases.

The fact that Python forces the developer to manually manage isolated directory symlinks (venvs) just to prevent local environment contamination — and that minor runtime mismatches can completely derail a standard onboarding experience — is a structural UX failure.

[andrewflnr]

> Zero. The required number of problems needs to be zero - hence my OP

This is unrealistic. You seem to have moved to Go as an alternative, but I know because I've seen the complaints that Go doesn't satisfy that standard either.

[awesome_dude]

I explictly say that NO dependency management is without problems, AND call out Go's attempts in my OP.

You came barrelling because your favorite tool is mentioned, and then, when you realize that it deserves the criticism you try and play that?

Maybe, just maybe, take an objective look at the real problem (dependency management) and recognise that, as stated, nobody has solved it.

[never_inline]

> we cannot "pin" versions

you can? that's why go.sum exists. you can also use the replace directive for more advanced scenarios.

[awesome_dude]

No - go.sum alerts you to the change - it doesn't prevent it.

replace directives are ok, but you need to look at why workspaces were invented to get an idea of their shortcomings (hint: people used to have a replace directive locally that they would accidentally push and that would break other peoples builds)

[corndoge]

Nix solved it. Languages could choose to adopt Nix as their packaging system.

[tadfisher]

It did and didn't. Nix tools for building language-specific packages almost always wrap the language build tool/package manager. This can be easy or hard, depending on how onerous the build tool is for vendoring libraries.

What Nix and build tools need to agree on is a specification or protocol for "building a software dependency tree". Like, I should be able to say 'builder = cargo' in a Nix derivation and Cargo should be able to pick up everything it needs from the build environment. Alas, there is simply far too much tied up in nixpkg's stdenv for this to be viable, so we have magic stdenv builder behavior via hooks when a build tool is included in nativeBuildInputs.

[awesome_dude]

I think one of the key problems too is that a system level dependency is managed by people dedicated to ensuring the chaotic nature of the package they are responsible for conforms to the way the OS they are maintaining for has proscribed.

There's no real way to do that at a language level - we cannot have "Go has determined the package you are trying to fix has not met the versioning requirements proscribed so you cannot submit the patch to fix it"

What language dependencies do is what OSes would think of as "unofficial versioning" that is, an OS will let you install and run an unofficial version of some lib (we've all been there, right, multiple versions of some core library because one doesn't work with whatever you are trying to install), but they will not manage it at all.

[corndoge]

Thanks for writing this, I learned something

[gitaarik]

In theory, but not in practice