[snowycat]

Really impressive project! How did you learn all the things that you needed to make this? Are there any particular books or resources you would recommend? Also, how much did this entire project cost? I'd imagine processing that much data and storing it wasn't cheap.

[leijurv]

Mostly learned as I went! I had never before implemeneted Monte Carlo localization, or DBSCAN, or a disjoint union find; it was a lot of research and trial and error. I'd honestly recommend Wikipedia! I also read a lot of blog posts such as on data clustering in order to decide how to write an algorithm to tell what is and isn't a single contiguous base.

The main expense was priority queue, which is a $20/account/month payment to 2b2t to be able to join the server faster. Probably nearing a thousand dollars on that. We lightened this by creating a proxy system where other builders could join through our headless, and it would add nocom exploit packets from their account to 2b2t, and peel off the responses on the way back. They got a more reliable experience with less disconnects, and we got some extra capacity for free (completely unbeknownst to them of course haha). By the end, all the overworld accounts were being paid for and used by other people, I only had to cover the 2 accounts in the nether.

About $300 on a new 2TB nvme drive for postgres. An unknown amount on electricity. Other than that, all the big processing ran on an existing homelab server (thanks to fr1kin). About $150 on digitalocean for a droplet to run the headless minecraft with minimal ping to 2b2t ($10/month). Backups were just WAL rsyncs to postgres replicas ran (again on homelabs) by myself and a few others.