|
I wrote about some of these issues client-side, in a previous post about a Rust metaverse client. This is a much worse problem in a metaverse system, because there are no static game level maps. Every object in the world is in a database somewhere. Second Life / Open Simulator makes a big distinction between assets, inventories and area state. Assets (meshes, textures, animations, sounds) are immutable, and are stored more or less permanently. (There's a garbage collection batch job that runs monthly or so) Those are basically files. There's a vanilla web service running on an AWS web server, and Akamai, both heavily cached. Inventories are like file directories. They have asset UUIDs and some metadata (name, etc.) Those are in a database, but that data is dynamic and not cached. Each user has an inventory, of course, and it can be huge. 50,000 items are not unheard of. This is a metaverse; you can build stuff. Area state is in server memory for each region. That's saved periodically, once a minute or so. This is a backup file, not a database. If you wanted a more continuous save process, you could keep a log of recent changes on a different machine than the server. After a crash, reload the server state and rerun the recent changes. Area state is under a gigabyte per region (A region is 256x256 meters). With a three level system like this, none of the levels are severely overloaded. The greatest data volume is from the asset store, and because that's immutable, it can be and is cached extensively. There are three levels of caches - asset server, CDN, and client. It's still a problem getting assets out to the clients fast enough, but with prioritization and concurrency, that's solveable. The inventory database is mostly-read, so the usual scaling techniques for mostly-read databases work. Area state is in memory. The main trick is taking a clean backup without visibly freezing the system. |