[dasz]

https://wiki.python.org/moin/StacklessPython

Stackless Python allows you to run hundreds of thousands of tiny tasks, called "tasklets", in a single main thread. These tasklets can run completely decoupled, or they can communicate via "channels".

[topspin]

Stackless Python has also been a problem for EVE. Due to the cooperative multitasking of stackless tasklets the main loop of EVE can't leverage SMP. They've coped with this mainly in three ways. One is offloading as much as possible (network communication, for example) onto other threads. Another is "time dilation" where each 'tick' of the main loop is given more real time to process. This means the game runs slower, sometimes to an absurd degree in large engagements. Finally, there are hard limits on the number of players that can be present in a "system." The limits vary depending on what hardware a given system runs. On occasion these limits are too low for the number of players that might otherwise participate, which makes it a tactical matter that players actively exploit.

Seems like an unfortunate choice in a world where compute cores proliferate but clock rates increase only incrementally.

The original designers made this choice because they believed it reduced developer burden; they could deliver the game faster because the programming model is simplified.

I have been haunted by dreams of implementing a game engine to 'solve' EVE for about 15 years now. One imagines perhaps golang and it's goroutines efficiently distributing tens of thousands of players and other entities across a 64 core CPU... Or perhaps something akin to io_uring written in asynchronous Rust. I can't count the number of times I've diagrammed the memory structures and algorithms.

Amdahl's law is the ceiling of what is possible, but just approaching it would deliver staggering possibilities.

[NelsonMinar]

I suspect the Stackless choice made a lot more sense 20 years ago, before asyncore and when Twisted was brand new. And when CPUs had 2, maybe 4 cores and no one was running Python on more than one except for some very exotic stuff. Stackless seemed quite viable then.

What's surprising is they've never made a plan to migrate off. I'd like to think 95% of the code is platform agnostic and could run on any Python, but maybe it's not that simple and anyway that last 5% is awfully scary. Also perhaps Stackless works just fine for them, particularly since they've invested in maintaining it. The single core thing is definitely a problem but there's no magic bullet to parallelize code like that in any Python variant.

I wonder if they ever considered Erlang instead of Python?

[topspin]

I used the word "unfortunate" because it wasn't necessarily wrong or foolish at the time. There were still significant clock rate increases expected in future devices. We used to think 10 GHz and more was right around the corner. Instead clock rate increases became incremental due to the inherent properties of complex digital semiconductors, which isn't something one should expect EVE designers to have predicted.

The game is old and the game model is well understood. It would take some hard thinking but I suspect it would be possible to rework the core game loop with a reasonable amount of effort. However, I don't think the motivation exists now; the game was sold by the original developers (CCP) and the new owners appear happy with the steady revenue of EVE's modest but loyal paying accounts. The developer would have to imagine that enabling the game to handle one to two orders of magnitude more players in a battle would translate into a substantially larger player base.

Having studied the problem for a long time I believe the next bottleneck, after the processing of game entities is fully parallelized, would be communication. Keeping all the clients aligned with the current state of, say, 100,000+ entities would be challenging. It would require a sophisticated pipeline and perhaps something akin to streaming, ah la H.264. In any case the communication solution would require a corresponding orders of magnitude decrease in the rate of data sent to clients. Then there is the database which, last I heard, is Microsoft SQL Server...

It's a fascinating problem.