Hacker News new | ask | show | jobs
by crabbone 3 days ago
1. There are multiple levels at which things are hashed, signed and checked when it comes to Python packages. Eg. each file in the Wheel beside the RECORD file is hashed using SHA256. This hash is never checked :( You can also sign your packages (RECORD.jws anyone? Is that still supported?), but nobody checks that either.

2. There are hashes in the HTML served by PyPI. These are updated at the whim of both the index and the publisher. Even though they are checked by pip during install, they are worthless.

3. There are many ways to install packages that work around (2). Custom index server doesn't have to provide hashes, and pip will happily install that. You can install from sources, from a package you've downloaded somewhere, form VCS, you can build it during install, all without even prompting the user to confirm the very scary choices.

NB. I have no idea how do you make the leap from "adding files to release" to "not modifying the release". To me, adding file to release is sure as hell modifying it. Here's a very simple malicious example:

I release package "innocent" with an empty "scripts" section. Then, in the subsequent modification to this release, I add the "scripts" section with a script named "notebook". Now, whenever my user wants to run Jupyter notebook, they will call my "notebook" program, not the one from Jupyter package.

1 comments

There are a lot of misconceptions here:

1. “Wheel” signatures (e.g. RECORD.jws) were specified but never actually implemented in any major packaging installer, to the best of my knowledge. In that sense it’s not really meaningful to ask whether they’re “still” supported, since nothing ever did support them unfortunately. The reasons for that are multifaceted, but a single easily identifiable reason is that they don’t solve the key/identity distribution problem.

2. The primary benefit of index-supplied hashing is for lockfiles. Specifically, they make lockfiles like uv.lock and pylock.toml useful/meaningful.

3. Is the source distribution problem, and is sort of fundamental to any packaging system that has any amount of dynamism in it. There’s a good argument to be made for reducing unnecessary dynamism (e.g. via wheels), but a nontrivial percent of Python users actually want this behavior.

Re: files: I think you’re getting confused about distribution files (sdists and wheels) versus the individual contents therein. You can’t modify distribution files on PyPI, you can only add new, unique ones. And this is now restricted to 14 days for each release.

1. There was a discussion few years ago to remove the signatures from the format at all. I don't know what the resolution was, that's why I don't know if it's still supported, as in is it supported by the specification or not.

2. Lock files are hilariously wrong in Python. They don't accomplish what they claim because the only tool that in principle could do that and that is actually used with Python deployment / development is conda, and to the best of my knowledge conda's environment doesn't support locks. All other installers don't even guarantee that successfully running them will produce an environment with all dependencies met. But, not only that, they don't prevent users from becoming victims of manipulations that happen index-side because they don't store the checksums.

3. "nontrivial percent of Python users actually want this behavior." I'm yet to find one. Most Python users simply don't understand how any of their packaging works and come up with absurdly bad ideas and "solutions" to their problems. They may tell you they want something in the same sense as a drug addict may tell you they want more drugs, but the "actually" isn't there. They "actually" don't know what they want most of the time.

> You can’t modify distribution files on PyPI, you can only add new, unique ones.

I just gave an example of how this modifies the package. Not sure what problem do you see with it.