[epaik]

Looks pretty cool, although I don't know how useful this would be for games.

Isn't the only real bottleneck for large-scale real-time MMOs the network bandwidth needed between the server and client? While this tech would improve the efficiency of handling data on the server, it wouldn't be able to solve the inherent problem of network limitations.

Major props for making it open source though, I'll enjoy looking at the code.

[pron]

Fact of the matter is that, other than EVE online, there aren't any "large-scale" MMOs out there. Most of them are limited to one server per world/shard/instance (and even EVE is limited to one server per solar-system AFAIK), and the number of concurrent players that entails. This is a first step in helping them build bigger worlds with more players, that seamlessly scale over a cluster.

[jff]

IIRC they can run several solar systems on a server, but they have devoted one whole server to the unofficial trading hub, Jita, because it is so busy.

EVE has the nice property where the universe is already split up into these nice partitions connected by a network of jump gates. It maps very nicely onto a server farm.

[epaik]

You're right that there aren't many truly large-scale seamless MMOs out there, and I agree that this project would help servers scale their worlds.

Sorry if my previous post came off as dismissive, as a game development enthusiast I was focused only on the application to games rather than the technology itself. The point I was trying to make is that even with this technology, we are unfortunately not closer to being able to create real-time action mmos. For an advancement in this area, we need a leap in network capabilities, rather than server efficiency.

[Arelius]

Sharding like done in Eve is well established technically in the game development community. And I agree that this basically solves the problem we already know how to solve. The problem is in content, Most MMO's are designed to be content heavy, with zones of high-traffic, this causes a population density problem that creates the exact bandwidth problem you mention.

This doesn't solve any new problems, what it does do is provide a licensable technology that many studios have to re-invent themselves, there is value in that however.

[pron]

So it's a good thing, then, that, a) it's open-source! and b) it's not even directed specifically at games!

[Arelius]

I don't mean that it won't be useful in game development, just that I don't think it'll make games particularly more "massive".

[epaik]

Definitely a good thing! :)

[Arelius]

I'd argue that the lack of "large-scale" MMO's is more due content production problems, and activity density (I don't care how good this grid database is, there is a density that will bottleneck it, at the very least, it will bottleneck the client's bandwidth pipes if nothing else.) Rather than huge technical limitations. MMO developers are pretty familiar with sharding these days.

[rdtsc]

I thought of that too. So if you have a large MMO presumable players wouldn't be uniformly distributed in the world. As in, here, there is a fairly constant 100 players per sq km. Rather I see hubs forming (maybe with some power law distribution) -- maybe a large city, market, planetary system, ring of hell, a battlefield that would disproportion ally hold a large number of entities vs other ares.

There then instead of node ownership based on the space grid, you'd want to somehow have node ownership based on clustering/density.

So maybe constantly iterate a K-means clustering algorithm, where servers are cluster centers and every player/client in the cluster belongs to that server.

That would be my back of the envelope approach. It probably has lots of terrible flaws that I haven't thought of yet.

[Arelius]

I mean, I imagine that the system can handle the density by progressive balancing the tree. The problem is that gameplay doesn't do the same thing, A client wants to see all characters within 10(or w/e) meters, not the closest 10 characters.

[pron]

This has nothing to do with sharding (except that if games wanted to use this it might help them do away with shards), and not related particularly to games at all (in fact, we think it could also be great for real-time graph databases).

[Arelius]

I'm under the impression that the dependence on spacial locality in it's parallelization that it would actually perform pretty poorly as a general real-time graph database.

[sp332]

Sharding is a difficult design problem. It would be easier to make games (and nicer to play them) if you didn't have to shard the world up.

[daeken]

That's actually not true. A large part of the reason MMOs typically split the userbase into many shards is that the world design itself won't cope with the full userbase in a single instance of the world. You need a lot more space, a lot more places where interactions happen, etc. Imagine how much bigger the world of WoW would need to be to make the entire userbase work in a single world.

One of my favorite articles is on this very subject: http://www.gamasutra.com/view/news/114192/Opinion_Designing_...

[sp332]

I know. Deciding where to make the boundaries, and actually handling the handoffs between shards is difficult. It would be easier to write a game if you didn't have to worry about sharding.

[daeken]

I think you're misunderstanding what shards are. Shards in this context aren't pieces that interconnect, but rather completely independent worlds. A 'shard' is the entire world where players exist; players from shard A will never see players from shard B.

[sp332]

Oh, I've been referring to those as instances. Did I have the wrong term?

[lallysingh]

I think you mean latency between client and server. Second, there's a small exponent (let's say 1.3) in your simulation work (collision detection and newtonian motion), that can saturate a CPU quickly. quickly

Your end-to-end latency from a user clicking something and seeing the results is 2x latency + simulation time.

While I like software implementations of hardware optimizations as much as the next guy, what kind of replication distribution do you expect? You're splitting up your data-set in RAM across many machines, yet you've got them replicating data from each other. How much of your data-set do you expect to be accessing frequently?

[pron]

Each item can be replicated to many machines, but written by one machine at a time. Galaxy works best when items that are replicated to many nodes are updated infrequently. This works very well when distributing data structures like B-trees, when the root of the tree is read by all but updated rarely, while the leaves remain pretty much confined to only one node and are updated regularly.

This is all for latency reasons. Fault-tolerance is a different story.

[lallysingh]

Ok. Makes perfect sense for B-Trees (as long as you have enough remaining memory/machine for a decent size cache). I think for a game's partitioned scene graph, the change rate would be rather painful.

[wmf]

It's useful for games that aren't possible today.