Hacker News new | ask | show | jobs
by sebstefan 500 days ago
So it also dumps functions and is able to import them back?

Does the function still need to be in memory to be loaded again ("does it just dump the pointer") or can I save it to disk, shut off the interpreter, boot it again and it imports it fine (in which case it somehow dumps them as code...?)?

Even in the linked test case on the readme you don't show the output/expectation of the serialization

3 comments

> can I save it to disk, shut off the interpreter, boot it again and it imports it fine (in which case it somehow dumps them as code...?

Yes, it dumps them as bytecode (probably not compatible between completely different interpreters).

It even preserves debug metadata, so stack traces involving serialized/deserialized functions look right, and still show the original source file.

This is really neat.

Thank you, it is really nice to hear. Though, I have to give credit to Lua's standard library -- the basic function serialization (without upvalues) is implemented there as `string.dump`.
Be aware that you're gonna have a bad time in scenarios where code is serialized using one Lua version and deserialized using another. Bytecode compatibility is not guaranteed between different versions of Lua(JIT).

I've shipped Love2D games as bytecode that wouldn't run on many Linux boxes because their LuaJIT installation (which is not part of Love2D but part of the system) was too old, or they stopped working after the user updated their system. There's a plethora of situations where something like that can happen.

I'm also wary of the "upvalues are preserved" feature, which sounds like a huge footgun, but I haven't looked into the details of your implementation.

Functions are apparently serialized as a bytecode dump contained in a self-extracting expression. So everything can indeed be serialized as a Lua expression. Seems that the author also tried to preserve as many upvalues as possible, though I feel that is way more dangerous than I would like.
Yep, that is correct. I think ldump is able to preserve all upvalues, even on edge cases such as "_ENV" and joined upvalues (multiple functions referencing one upvalue). A closure is basically an object with a single method and upvalues as fields -- serialization is straightforward. I think I got it covered, but I would be glad to hear ideas about where the serialization can be unstable.
The function (even a closure) would be fully recreated on deserialization, it is fully safe to save it to disk. It wouldn't preserve reference equality -- it would be a new function -- but the behaviour and the state (if using closures) would be equivalent.

I didn't include asserts in the linked case, because I thought it would be too verbose. You can see asserts in the test, that is linked below the example. Maybe it was the wrong call, I will think about including asserts into the example itself.

That's super cool

I think you could make it clearer, try reading the readme as someone with the preconceived notion that this is Yet Another Lua Serializer that translates functions, userdata and threads to their tostring() output. There are hundreds of those projects