[Animats]

One tiny design error had huge implications. The UDP messages are multiple messages per packet with the form

    [msg_type variable_length_message]

There is no message length in the message. So, to parse the message stream, the receiver must know how to parse each message type in detail, and can't skip message types it doesn't understand. Thus, no new message types can be sent until everything that receives them has a parser for the new type. And, so, there have been no new message types since about 2016.

Most game systems use

    [msg_type msg_length variable_length_message]

so you can introduce new messages and not break old receivers.

[sgtnoodle]

Seems like a good case for protobufs. Maybe there weren't any good IDLs back when they originated the protocol, though.

It seems like there's an easy way to retrofit the protocol without breaking it for older clients (as long as older clients truncate packets once a message type is unknown). Put a new message type at the end of the packet, with a list of message start offsets in it. By including its own offset as the last element, the start of the list is easily found. The first message offset could be omitted, since it is implicitly zero.

[n42]

I am thoroughly enjoying all your contributions in this thread. I love getting a look into the architectural bits of a project/community of this scale that I don’t really know much about. Very interesting, thanks

[Animats]

As I mentioned previously, there's not much written about this. Which is probably why existing metaverse projects are so awful. It's mostly taking Unity or Unreal Engine, which are intended for use with carefully pre-built content, and somehow trying to make a large dynamic virtual world from those parts. The duct tape is troublesome.

The primary technical designer of Second Life did a very good job. Then he had a disagreement with management and was fired. So he went to Facebook, developed their mobile client, became a Facebook VP, made lots of money, and was semi-retired for a while.

Scaling to a big world is really hard. Big has two dimensions - area and density. The Second Life architecture scales well in area but not in population density. More than 20-30 users in a region will choke it. More are possible if most users sit down and doesn't move, because sitters are not getting physical simulation. So audience-type events work.

Improbable was working on large crowds, but their solution is really expensive to run. Which is why they just do demos of Otherside for a few hours at a time every few months. Also, they use very simple avatars and do not, as yet, support vehicles. Took them over US$500 million to get to that point. They have dynamic regions - more people, divide the world into smaller chunks. That introduces a whole range of new problems. What if I have 20 people on my boat, and a region boundary moves under the boat while I'm crossing it? Stuff like that. Fixed region crossings involving multiple avatars and vehicles are troublesome in Second Life. People do try bus tours, but sometimes an avatar gets Left Behind at a region crossing. Linden Lab's devs have been trying to fix that for at least 8 years, without success. It's a tough problem in real time distributed system design.

[mnahkies]

What's stopping them from introducing a new major version of the protocol that addresses this?

Presumably they could have a negotiation process that newer clients invoke to upgrade the message stream to a new format, and support both in parallel on the backend until an insignificant number of sessions were using the old version.