[YetAnotherNick]

I just don't understand how these guys could literally give terrabytes of free storage and free data transfer to everyone. I was doing some calculation of cost from my storage and transfers and if they used something like S3 it would costed them 1000s of dollar. And I don't pay them anything.

[mritchie712]

> As Hugging Face hosts nearly 11PB of datasets with Parquet files alone accounting for over 2.2PB of that storage

11PB on S3 would cost ~$250k per month / $3m per year.

HuggingFace has raised almost $400M.

Not saying it's nothing, but probably not a big deal to them (e.g. ~10 of their 400+ staff cost more).

[fpgaminer]

HuggingFace really is such an amazing resource to the ML community. Not just for storing datasets, but being able to stand up a demo of my models using spaces for anyone to use? It's hard to overstate how useful that is.

[ylow]

We are here to help lower that :-) . As we can push dedupe to the edge we can save on bandwidth as well. And hopefully make everyone upload and download faster.