[justinclift]

Wow. ~US$40k/mo running costs, with about US$5k/mo for server hosting:

https://lichess.org/costs

It looks like the servers are individually managed via OVH or similar, rather than running their own gear in co-location or similar. Wonder why?

[tormeh]

Easy: If something is wrong with the physical gear it's OVH's problem rather than theirs. It also means no one has to ever go to the data center which is probably important for a geographically distributed team (I assume they are). Cheap, no-frills cloud is extremely underrated, IMO.

[danpalmer]

Underrated? The flip side is that hardware failures are still your problem like they would be with rolling your own hosting. I think they’re correctly rated for the position on the scale of traders that they provide.

[squigz]

Surprising numbers, and really goes to show how cheap the hardware/software side is for this sort of thing if you do it right.

I wonder what the "Misc dev salaries" is for - only curious because it's a flat $5k

[justinclift]

Heh heh heh.

To me those numbers seem on the high side as I'm (personally) used to (for cheap projects) scavenging together stuff from Ebay before deploying to a data centre. ;)

[squigz]

lichess is hardly a "cheap project" though :P It's one of the most popular chess platforms

[justinclift]

Sure, but they seem to be extremely budget constrained. ;)

[me_me_me]

no surprise there tbh

Here is a comparison of free and their premium accounts:

https://lichess.org/features

[justinclift]

Looks like they're fulfilling their mission?

[benmmurphy]

its also crazy how much cheaper it is than AWS. the primary DB is around $500/month with 32 CPU and 256 GB of RAM and 7TB. AWS RDS db.m6gd.8xlarge which is 32 CPU and 128 GB of RAM costs $2150/month before paying for storage as well.

[bryan_w]

Yeah, but you get what you pay for. That m6gd.8xlarge would never be subject to such a long network outage as once the hardware fault was detected, it would be moved to another machine

[beaviskhan]

Yup, and you also get to make AWS deal with OS upgrades, DB upgrades, backups, etc.

[paulryanrogers]

You have to pay 2x for multi-AZ or you get downtime for upgrades. And DB major version upgrades require manual effort unless you want to roll the dice on their new blue-green feature, which can take hours to fail or finish cutting over.

[aflukasz]

> You have to pay 2x for multi-AZ or you get downtime for upgrades.

Worse. In Single AZ deployments you get (short, but not that short or strongly bound) downtime for daily backups and when doing snapshots. Source:

- https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_...: "During the automatic backup window, storage I/O might be suspended briefly while the backup process initializes (typically under a few seconds). [...] For MariaDB, MySQL, Oracle, and PostgreSQL, I/O activity isn't suspended on your primary during backup for Multi-AZ deployments because the backup is taken from the standby. ",

- https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_...: "Amazon RDS creates a storage volume snapshot of your DB instance, backing up the entire DB instance and not just individual databases. Creating this DB snapshot on a Single-AZ DB instance results in a brief I/O suspension that can last from a few seconds to a few minutes, depending on the size and class of your DB instance.".

Not to mention that multi-AZ deployments incur extra transfer cost between zones - not between DB instances (this one is free, last time I checked), but between your compute deployments and DB instances, if your compute does not automatically follow the zone of the db host it talks to.

[chmod775]

Given that the full AWS setup that would replace that one server would cost closer to $6000-8000 / month, they could just use that money to buy a bunch of extra hard drives, a backup server, and hire a junior dev/sysadmin whose only job is to watch over it, still coming out ahead of AWS.

[acdha]

I’m seeing more like $5k/mo with reservations but even at those figures … how many skilled DBAs are you getting for $72-96k? Don’t forget that rolling it yourself means you have to build and test all of the hardware, maintenance processes, backups, multi-data center HA, etc. yourself. That’s not junior trainee level work and some of it is ongoing (e.g. every OS and hardware change) or at intervals not of your choosing (say you discover a kernel or firmware issue when it’s crashing randomly - how many months of savings will be canceled out by pulling senior people off of whatever they’re working on?).

You can beat AWS on pricing but not like this. You need to be finding areas where you have a lot of baseline demand – enough to amortize the cost of all of the lower level work – and can cut some of the things they do which you don’t need. For example, if you can afford more downtime in a disaster scenario or can rely on an external rebuild process if the database backups turn out to be unusable.