[ryandrake]

> Parent is right, a bunch of them basically are a real-time OS sitting on a small amount of the underlying OS (if one exists). When you have to write your own malloc() and fopen(), and your own thread scheduling,

I don't doubt that game developers do this, but I've never really heard a satisfying reason why, besides vague hand-waving about "needing more performance". I'm not a game developer though, so maybe it's truly needed. Or maybe it was needed in the 80s and 90s but not anymore, and maybe the mentality is simply stuck in the minds of game developers.

I remember interviewing a former game developer at a [not game] company, and we started talking about the C++ Standard Library. Candidate insisted that "the STL is slow," and that you shouldn't use it in any production software, and that if he worked on a project that used it, the first thing he'd want to do was re-write it using his own containers and allocators and so on. I would ask him questions like "what is slow about it" and "how do you know where it's slow, as used in our company's application" and "have you profiled any code that used it to find out where it's slow" and so on, and it became clear (to me) that he was probably just operating out of some ancient tribal believe that "STL = slow" that was organically passed on to him from other game developers throughout the decades.

[Mabusto]

I haven't worked on games in about 10 years, but I wrote core tech for AAA games during the 2000's. Stuff like async I/O, reflection, memory allocation, low level networking, and especially containers were all written in-house. Quite a bit of it was utilitarian, but I agree, a lot was a case of NIH syndrome. Some studios had these insane container libraries that just archived whatever weird data structure someone had as a toy project in university. Math libraries were the same. Scripting languages were all half-baked toy projects by the lead engineer etc.

For consoles, it wasn't that the STL was slow, it was that not all of it was fast, and some parts really were just trash written by whatever the compiler vendor had put together. RTTI was also generally not allowed as it was "slow", and virtual functions weren't allowed at one place because the vtable took up "a ton of extra space". So some of it was cultural, some of it was advice that was ok-ish maybe 15 years before the project began, and a lot of it was just not understanding what was happening under the hood.

[dahart]

Custom allocators were needed, and often still are, in order to have enough memory in the first place (for example memory pools for specific sized small allocations are common), to avoid fragmentation that could lead to out-of-memory conditions or the need to defrag over time, to have control over how the free list regains memory and how long it takes. It’s important in a real-time or embedded application to prioritize predictability over some of the host OS’s goals, and to have stronger guarantees.

IIRC I think one of the primary reasons the EASTL was developed is because the STL at the time did not allow overriding the default built-in memory allocator, and because STL did a bunch of memory allocation.

This is changing and games use lots more STL and dynamic memory allocation than they used to, but a decade ago and earlier, STL use was generally discouraged for the same reason that any and all heap allocations were discouraged for gameplay and engine code: because doing heap allocations during real-time and in performance critical sections can impact performance significantly, and can be dangerous. This is still very true today: the first rule of high performance is to avoid dynamic memory allocations in the inner loop -- true in JavaScript, CUDA, C++, even Python. Dynamic memory allocations can sneak up on you too. It might seem small or innocuous until it piles up or pushes you over a threshold or hits some kind of lock condition.

One mentality I learned game programming that can be counter-intuitive at first is how and when to reserve the maximum amount of memory you need for a given feature and never change it. It can feel at first like you’re hogging way too much, or that something’s wrong with coding that way, but it’s much more predictable, can be much higher performance, and it can be safer for the overall system to ensure that you will never have a weird condition where several things all need more memory than expected at the same time and it suddenly crashes.

[f33d5173]

I'm not a game developer, but I was similarly lead to believe that stl containers were especially slow compared to other implementations. This is supposed to be because the stl has more restrictions on the implementation owing to the interface they have to export. The stl hash table for example has to provide low level access to the buckets, even though modern hash tables usually avoid having buckets in the first place. By lifting these restrictions the performance can be improved tremendously. Facebook, for example, are supposed to have their own in house hash table implementation to achieve this.

[kamikaz1k]

That’s a great example to raise, and I think you’re right to point it out.

Any field is susceptible to cargo-culting, especially ppl who are just trying to get their job done so they focus on having to rediscover everything their leaders are claiming to have verified.

But for the specific claim you’re challenging, it’s hard to really convey the realities without having some experience in those problems.

High level it really comes down to:

1. Relaxing some generic constraints to apply game specific ones

2. Fine grained control over execution, again by relaxing some constraints

3. Game programming can play fast and loose with a lot of things for the sake of speed; this is not generalizable

[badsectoracula]

The problem with STL is that there isn't one of it but every compiler/C++ library/platform has its own and some are better than others. Writing your own containers at least means you take one uncertain dependency out of the scene for something that'd be all over the engine everywhere (so it isn't easy to replace later).

Though personally i'd make my own containers because i heavily dislike the API design C++ library has and i am glad every single engine i've worked on had their own (well, except in a single case where while the engine did use its own containers it was basically 99.9% the same API design as the C++ library :-P).