[vvanders]

They get close but I think there's a few restrictions(rightly so from a safety perspective) that keep untyped arena allocators and in-place loading from working(see FlatBuffers for a fantastic library for in-place loading).

Edit(since I can't reply yet to child):

By in-place loading I don't mean placement new(which untyped arena covers). Usually in-place loading means constructing an object(usually just by reinterpret_cast<T*>(data)) over a block of memory read in from disk/network such that all data members "line up" with data as it was serialized.

This also includes the ability to mutate the in-place data but have initialized to some sane values. 99% of the time this is paired with either some sort of preprocessor(usually written in the same compiler as the reader to guarantee data matching).

Quite a few games use this technique to allocate levels/fixed entities such that once you've pulled them disk you just need to do some trivial pointer fixup and away you go. There's some pretty incredible speed increases(100-200x) but obviously it's incredibly fragile and complex to get right.

Flatbuffers does a good job of trading off fragility for usability by encoding offsets in the placement data. Reading data is slightly slower due to needing to calculate the offset into the data but on the plus side it doesn't depend on compiler specific layouts :).

[steveklabnik]

By "in-place loading" do you mean placement new?

And while Rust does have restrictions for safety, unsafe should give you the power to do anything equivalent C or C++ can do. Of course, many idioms are different, especially for safe Rust.

[steveklabnik]

  > Edit(since I can't reply yet to child):

HN #protip: if you click on comment link itself, it will let you comment earlier.

Ah, I see. This isn't my area of specialty, but it _sounds_ like what Cap'n Proto does? Anyway, it doesn't sound impossible, though it might not be the most ergonomic. I'll check out that library, thanks.

[vvanders]

Learn something new every day!

Yeah, Cap'n Proto is very similar, FlatBuffers has the advantage in that it compiles against almost all versions of Visual Studio where Cap'n Proto unfortunately decided to use some C++14 features.

FWIW I think I've seen some ports of FlatBuffers into Rust since it's much easier to handle due to the offsets and translation on return value. The former(matching compiler layout) was the part that I was mentioning that seems like something a bit gnarlier than rust would like.

[steveklabnik]

Gotcha. Yeah, so the Rust Capn' Proto does compile-time codegen to write out all the structures, so the grossness is hidden away, or at least, you don't have to deal with writing it out. I believe it's basically the same as the C++ version.