[jshen]

" Sites that are read-only or store blob data in something like S3 can often avoid sharding"

Still too broad. Sorry, this is a pet peeve of mine, where tech people assume that everyone else has the same issues as them. For example, I worked on an ecommerce site that made over a mil a year. They had less than 10k products, and will never need sharding. They are not read-only, they have people updating their products on a daily basis through the site.

[nsanch]

Sorry, those were meant to be examples, not an exhaustive list.

[jshen]

Let me flip it around. You said, "I wouldn't build a serious site on top of anything that didn't have some sort of built-in sharding story".

I think the opposite. Most serious sites will never need sharding.

[nsanch]

My definition of seriousness includes some relatively large scale. Your definition of seriousness appears to mean any site that is important to the person or business running it. Is that a fair assessment?

I think your definition is better, and I should've said "a potentially-large site" or something like that.

[jshen]

Yes, this. Thanks for listening. It's a pet peeve of mine because most people work on serious sites, and a minority of them have large scale data, but people often imply that everyone needs to solve massive scaling problems even if they will never have them.

[leothekim]

What do you mean by "serious"? That's broadly stated as well.

[jshen]

techcrunch is a serious site. They don't need sharding. They have thousands of posts, not millions, and certainly not billions or trillions.

[taligent]

You're joking, right ?

Every large site on the internet has talked about strategies around sharding. At some point you are surely going to hit the physical limitations of one database on one server.

[jshen]

He said "serious site". He did not say "large site". For example, techcrunch is most certainly a serious site. It is NOT large scale and in need of sharding.