Hacker News new | ask | show | jobs
by autarch 2053 days ago
This article ignores the fact that for many languages, the package repository is maintained as free software by volunteers (sometimes with funding from a foundation). This includes Perl, Python, Ruby, Rust, and many others.

NPM is the odd one out, really. I don't think letting one company control a language ecosystem's single package registry is a great idea, for all the reasons that the author notes!
1 comments
takluyver 2053 days ago
I agree, though it's worth noting that while volunteers can maintain the software and administer the indexes, they also rely on infrastructure provided by big corporations. E.g. the Python Package Index runs on a CDN provided by fastly, which serves hundreds of TB per day. I very much doubt the non-profit Python Software Foundation could afford that bandwidth if it wasn't an in-kind donation.
link
luhn 2053 days ago
It's a couple petabytes, Michael. What could it cost, $10?

Seriously though, Fastly's donation of their CDN service is generous and eases the burden on the PSF, but if push came to shove they could definitely afford the bandwidth. In 2018 they had a net income of half a million.

link
di 2052 days ago
Hi, PyPI maintainer and PSF director here.

There's absolutely no way the PSF could afford PyPI's bandwidth out of pocket. Last I checked our "bill" from Fastly would be close to $1.5M/month.

Also given that PyPI is critical infrastructure for millions of people and software projects, anything cheaper would not really cut it.

link
tedivm 2052 days ago
As of March PyPi was pushing out 300TB a day through its CDN. Ignoring the "off the shelf price" of $0.12/gb and assuming they negotiate a bulk discount driving them down to $0.05/gb that's still $15,000 a day (or just shy of $5.5 million a year). Their net income in 2018 would cover less than 10% of that bill.
link
true_religion 2052 days ago
Well.... they couldn't afford Fast.ly, or Akamai, or Brightcove, but they could afford any tier-2 or tier-3 CDN.

For example CDN77, would start you off at 0.016/gb if you have more than 100TB per month, without any negotiation.

If you have 300TB per day, you surely can negotiate sub-cent pricing somewhere.

No, the costs are manageable.

I think the charitable/community projects setup for developers, need to act like non-profits in other sectors and actively seek out donations.

link
takluyver 2052 days ago
I assume there's some difference in e.g. speed or reliability between Fastly (8 c/GB) and CDN77 (1.6 c/GB)? Even if you can negotiate it to 0.8 c/GB, you're still talking about a huge piece of the PSF's budget. They would have to either cut other expenditure (e.g. making grants) dramatically, or find a lot of new income (continuously, not just a one-off donation drive). And PyPI's bandwidth is growing rapidly [1].

If PyPI didn't have sponsor providing bandwidth, I'd guess it would implement some form of rate limiting and encourage people to mirror/cache packages much more to reduce load. I don't think it would die completely, but it would be less convenient and still cost the PSF a fair bit of money.

[1] https://twitter.com/di_codes/status/1235707819955032069

link
takluyver 2052 days ago
The PSF's tax returns are published on python.org, and 'revenue less expenses' for 2018 was just under $280k. IANA accountant, so maybe that's the wrong line to look at.
link
webology 2052 days ago
Hi, I'm a PSF Director and the PSF's Treasure as of this year. For transparency, our tax returns as of 2018 (and soon 2019) are up on https://www.python.org/psf/records/

You are right that this donation does not show up in our tax filings because they provide it to any OSS project.

link
autarch 2052 days ago
Also, it's not income that you would put in your 990s. Similarly, the donor cannot deduct the expense of the donation.
link
webology 2051 days ago
Correct on both points. (I do consider it a potential liability though.)
link
autarch 2053 days ago
Yes, that's a good point. The Perl repo and services are also relying on donations from various companies. That said, it's easier to switch donors than it is to switch repositories.
link
ForHackernews 2053 days ago
If necessary, it seems like it'd be easy for any of these package registries to ~~blackmail~~ encourage big companies into donating infrastructure.

"If you don't support us, we might accidentally forget to audit our packages and feed malware into your build pipelines. It'd truly be a shame..."

link