[scott_s]

Potentially. I was also skeptical of this four years ago, and said as much on here (https://news.ycombinator.com/item?id=22972538).

However, I then dug into it a bit. From my digging 4 years ago (https://news.ycombinator.com/item?id=23460200):

> I spent some time crawling through the proceedings of Very Large Databases (VLDB) and the ACM Digital Library, and I could find no instances of "shard" used to mean the partitioning of a database prior to 2001. (That paper is "Minerva: An automated resource provisioning tool for large-scale storage systems" in Transactions on Computer Systems, free-to-read at https://dl.acm.org/doi/abs/10.1145/502912.502915.)

> Other the other hand, I found many papers citing the SHARD paper - more than the official count. That's a difficulty with citation counts of old papers: a lot of the papers citing it are also old papers, and we're not consistent at tracking the citations of old papers. Personally, I don't have a conclusion. The SHARD paper is decently cited, and its usage is close to the modern one. On the other hand, I can't find any smoking gun pre-1997 usage of "shard" in the modern meaning.

I started my digging thinking I would quickly find a paper using "shard" in the modern database context that predated Ultima Online. I could not find it, so now I think it's plausible.

[dhosek]

I loved the casual comment that Flickr came out of a failed MMO (worth noting that the same team had another failed MMO that ended up spawning Slack!)

[selectodude]

You gotta feel for a guy who just really wants to make an awesome MMO but keeps fucking up and making shit like Flickr and Slack.

[otterley]

Yeah, the poor fella is crying all the way to the bank.

[andirk]

And maybe the opposite where the do-everything dream app of Twitter will eventually become an MMO.

[dhosek]

It kind of feels that way already.

[MichaelZuo]

A lot of innovations came from MMOs, so it shouldn’t be that surprising. Especially the big ones push the cutting edge in a way that no other applications do.

[jjice]

One MMO tech I always got a kick out of was WoW using Bit Torrent for updates at a time when it was massive and that bandwidth would have been brutal to output.

[mrsilencedogood]

"serve [seemingly] <50ms latency N:N coordination between thousands of clients, sometimes who are all literally in the same place (events, cities etc) expecting to observe their and other users actions ~immediately"

This still sounds like a really hard problem, I can't imagine trying to do it 20 years ago. (WoW turns 20 in Nov, and it wasn't the first, though perhaps the first to reach such supercilious scale?)

[pjc50]

It only really works because it isn't N:N, it's N:1:N, and it's aggressively sharded (you have to have a separate EU and US shard if you don't want one to have transatlantic latency, which is 50ms all on its own). While you can have a few thousand people in one place, the interactions are limited in ways which avoid N^2 blowups.

You'd be surprised what's possible with efficient in-memory client-server code.

[don_neufeld]

Yup, and there are some non- obvious ways to offload compute that were necessary back in those days.

Half the cluster for Planetside was edge servers whose entire job was to handle inbound packets and distribute them to the relevant other clients. One copy of each update was sent to the zone server, and it was then responsible for updating adjacency lists so the edge could do its job. This gave us much closer to linear rather than quadratic scaling. Even then the packet forwarding code in the edge servers didn’t do full packet decode, it just dug into the header deep enough to determine packet type, which allowed it to know how to forward.

I think we launched on ~20 servers per world, each of which was a dual core 300 MHz P3. That could handle 5000 players in a world. Including raytracing their projectiles against an 8km x 8km mesh ;)

I can’t even find processor benchmarks that far back, but I’d guess we ran the entire cluster on less CPU than most cell phones have today.

[astrobe_]

I've heard of tricks like player position prediction/interpolation by the clients, in particular for the FPS/Sim genres, which I guess is mainly aiming at mitigating "lag spikes" more than optimizing the network traffic, but it looks like it could give some leeway to servers as well ?

[pjc50]

Prediction lets you see through time but doesn't reduce the overall amount of work you have to do - the update confirming the prediction has to be sent eventually.

Also, remember that the server cannot afford to trust the clients about anything, not can it let them have too much information about the world.

[throwup238]

I think the Gamma Database Machine Project [1] demonstrated the technique in 1990, though it predates the word “shard.”

[1] https://pages.cs.wisc.edu/~dewitt/includes/paralleldb/ieee90...

[Suppafly]

Interesting, I had always assumed shard came from a specific company's technology and never gave it much thought. Kinda like how we talk about blade servers because of ibm's marketing.

[hedora]

Not sure about the word shard, but the 1401 used sharded storage in 1959:

https://en.m.wikipedia.org/wiki/IBM_1401

That was the best seller in history, at the time, but there are mechanical computers that played similar tricks going back to the early days of punch cards.

The Hollerith Tabulator won a competition in 1888.

https://www.census.gov/history/www/innovations/technology/th...

Anyway, yahoo probably sharded by the mid to late 1990s.

[scott_s]

Yes, the technique is obviously predated by Ultima Online. The question we're trying to answer is specifically about the etymology of sharding in the database context.