Hacker News new | ask | show | jobs
by amoshebb 636 days ago
Is it though? If it saves one engineer one afternoon that storage has paid for itself, and this thing has hundreds of thousands of downloads a day.

Wouldn’t it be more horrifying to force everybody who wants to use a prerelease to waste an afternoon getting it to build just to save half a hard drive?

1 comments

That's besides the point though. Yes, having prebuilt binaries is very helpful. But what happens if Fastly decides against renewing next time and there is nobody else willing to sponsor? The cost is through the sky for the PSF to handle. Where does PyPI go?
The issue there is how much gets downloaded from PyPI, not how much storage space it takes up. Making an archive copy of all of PyPI would only require a handful of ordinary drives. But the most popular packages get on the order of ten million daily downloads each.

When someone downloads and installs a package, it's generally a single wheel or sdist, out of a potentially massive version/platform matrix for that project. That inflates the storage cost, but it isn't why the bandwidth requirements are so high. A ton of CI systems are apparently poorly designed, individual wheels are bloated, and we can't use the best available compression. Those are the biggest issues.