[pdonis]

> There is constant breaking changes with it (both runtime and tooling).

I'm not sure what you mean. Python 2 to 3 was a breaking change, but that was just one change, not "constant breaking changes".

If you stick with one major version no old code breaks with a new minor version (e.g., you can run old 2.x code under 2.7 just fine, and you can run old 3.x code under 3.12 just fine). The minor version changes can add new features that your old code won't make use of (for example, old 3.x code won't use the "async" keyword or type annotations), but that doesn't make the old code break.

[bryancoxwell]

The Python 3.11 release notes have a pretty lengthy list of removed or changed Python and C APIs followed by guidance on porting to 3.11, which implies potentially breaking changes to me.

[pdonis]

It's a fair point that Python minor version changes can and do involve removal of previously deprecated APIs, which would break old code that used those APIs.

That said, when I look through the 3.11 release notes you refer to, I see basically three categories of such changes:

- Items that were deprecated very early in Python 3 development (3.2, for example). Since 3.3 was the first basically usable Python 3 version, I doubt there is much legacy Python 3 code that will be broken by these changes.

- Items related to early versions of new APIs introduced in Python 3 (for example, deprecating early versions of the async APIs now that async development has settled on later ones that were found to work better). These sorts of breaking changes can be avoided by not using APIs that are basically experimental and are declared to be so (as the early async APIs were).

- Items related to supporting old OSs or data formats that nobody really uses any more.

So, while these are, strictly speaking, breaking changes, I still don't think that "constant breaking changes" is a good description of the general state of Python development.

[jujube3]

Python's changes between releases are not limited to removing deprecated APIs. Sometimes semantics changes in breaking ways, or new reserved words crop up, etc. etc. It certainly is Russian roulette trying to run python code on any version other than the one it was written for.

[pdonis]

> Sometimes semantics changes in breaking ways, or new reserved words crop up, etc. etc.

Examples?

[mos_basik]

I know this specific example because it was one of a handful of reasons that delayed my workplace from upgrading past Python 3.5 (I think) for quite a while because of the addition of the reserved word `async`.

https://docs.python.org/3.11/whatsnew/3.5.html#pep-492-corou...

Can't speak to what GP was alluding to re: breaking semantic changes.

[InfamousRece]

For me switching to Python 3.11 was really tough because of various legacy stuff removals (like coroutine decorators etc). While my code did not use these, the dependencies did. For some dependencies I had to switch to different libraries altogether - and that required rewriting my code to work with them.

There was also some time in the past when async became a keyword. It turned out many packages had variables named async and that caused quite a bit of pain too.

[davedx]

The problem is `requirements.txt` doesn't do anything with downstream dependencies. There's nothing like a shinkwrap/lockfile in python. Even if you pin dependencies to exact versions, if you check your project out in a new environment and run pip install -r requirements.txt, you can end up with different, broken downstream dependencies.

This happens a lot for me.

[adamckay]

If you want to stick with using `pip` over any of the newer tools that build on top of it (Poetry - my favourite, pdm, pipenv, rye, ...) the simplest way I used in the past was to use a `requirements.human.txt` to set my dependencies, then install them in a venv and do `pip freeze > requirements.txt` to lock all of the transitive dependencies.

[baq]

That's an awareness problem. requirements.txt was invented... a long time ago, I think before the much more sane (but still not perfect) dependencies/lockfile split got popular. requirements.txt tries to be both - and it can be both, just not at the same time.

In short, you want your deployed software to use pip freeze > requirements.txt and libraries to only specify dependencies with minimal version conditions.

[davedx]

I did not know about pip freeze, doh. Thanks will check that out!

Edit: so if I understand it, this is just listing all packages in the current python env and writing them to a file. Hm, requires more discipline than the npm equivalent. But thats a natural consequence of pip defaulting to installing packages globally (vs npm which installs in local node_modules by default). Better but still not awesome IMO

[baq]

That’s why most folks recommend poetry or anything else made in the past decade. pip can work, it just doesn’t have sane (for 2020s) defaults.

[Too]

Completely false. Use pip freeze and pip install -c.

It’s one command more than npm install but that doesn’t mean it’s not there.

[lf-non]

> There's nothing like a shinkwrap/lockfile in python

Use poetry? I don't program in python regularly but looking at the github repo it seems actively maintained and quite popular.

https://python-poetry.org

[paleface]

> Use poetry?

Why not simply use something stable?!

I personally don’t understand why people think such glib, throwaway comments, are helpful. They always strike me, as lacking any foresight.

How many abstractions, on the core tool, are required, to force its stability over time? What happens if poetry introduces breaking changes?

[lf-non]

Why would you bluntly assume my comment lacks any foresight? I was simply recommending you a tool that I used, albeit briefly, that solves the exact the same problem for which you are claiming no solution exists.

Nobody is denying that it would be ideal if there is one best solution to every problem in the ecosystem. But at the end of the day all software, including core and third party libs is just code written by people, and it is too much to expect that any person (or a group of them) gets everything right the first time. Change, breaking or otherwise, is inevitable as people learn from their mistakes - its not like the core is guaranteed to never have any breaking changes either.

Just like you can pin the version of libraries, you can pin the versions of your tools too, as long as they are not depending on external services with no versioning. The point of the post is not absolute avoidance of change. It is to opt into a workflow and tooling setup so you can deal with the upstream changes at your own time and convenience.

And BTW, looking at their versioning, poetry hasn't yet had any breaking changes in its 4+ years of existence.

[toasted-subs]

Try poetry, I like it a lot more than conda.

The basic virtual environments work excellently too.

[lmm]

> you can run old 2.x code under 2.7 just fine

No you can't lol. There were major breaking changes from 2.4 -> 2.5, and smaller but still breaking ones for 2.5 -> 2.6.

[pdonis]

> There were major breaking changes from 2.4 -> 2.5, and smaller but still breaking ones for 2.5 -> 2.6.

Such as?

[lmm]

I don't remember the specifics, but I spent at least a few weeks early in my career fixing a Python 2.3 codebase to run on 2.4.

[pdonis]

In the GP you said 2.5 and 2.6, not 2.4.

That said, I remember all three of those transitions (2.3 to 2.4, 2.4 to 2.5, and 2.5 to 2.6), and I remember changing Python code to make use of new features introduced in those transitions (for example, using with statements and context managers in 2.5), but those aren't breaking changes; the old code still worked, it just wasn't as robust as using the new features.

[lmm]

> In the GP you said 2.5 and 2.6, not 2.4.

Sorry, yes, fixing a 2.4 codebase to run on 2.5. It was a while ago.

[wruza]

Something called onnx (all iirc) requires Python 3.8-3.9 but not 3.10+ in which it doesn’t work. So for my various AI needs I have three versions of Python 3 installed through different channels. And of course they all have their own multi-gigabyte caches of base libraries and models.

I know it may be more complex or trivial than I think, or tied to very few specific packages, but that’s the point – I have to figure it out where I shouldn’t need to. In contrast, I’m sure that no matter which latest version of Node I have, it will work.

I mean I was, up until Node 19/20, where they broke the loader, so ts-node doesn’t work anymore and the suggestion is to re-learn something called tsx. F that nonsense.