Hacker News new | ask | show | jobs
by JonChesterfield 833 days ago
> You cannot pass functions, promises, or other non-serializable objects

Functions are serializable. And deserializable. You can absolutely take one defined on one machine, kick it over the wire, then execute it on another one. Likewise closures, you serialise the associated state as well.

Promises are the same thing - a distributed language runtime should be exactly the sort of graph execution system which lets you pass promises between machines and have things work out.

Further, they don't say what other "non-serializable" things are. Is a DAG serialisable without spuriously unrolling it into a tree and failing to fold back into a DAG at the other end? How about a graph? Given that all things are serializable it is difficult to guess what things they haven't implemented.

So by "they've got the implementation wrong", I'm saying they haven't implemented the tricky part of serialisation, only the copy ints over wires part. And by "wrong", I'm saying that a language runtime dedicated to transparently running typescript functions on other machines should be able to run those functions on other machines even if they have closed over state.

1 comments

Author here. Absolutely agree with what you're saying.

Maybe the line could be written better. It is supposed to say: You can't pass functions and promises. And you can't pass in other data types that are not serialisable.

I definitely did not intend to say that functions or promises are not serialisable, and it is something we're looking into. The complexity is with serialising the closure state.

The other thing we're implementing is constraining the data types via the type system itself.

edit: punctuation

All data is serialisable. I'd say that's axiomatic. Data is information with a concrete representation.

At present the docs say it can serialise things that it can serialise with a couple of examples on each side.

My best guess is there's some limitations caused by the stack under your implementation that you haven't worked around, but I don't know what those limitations are or which ones you plan to resolve.

In addition to what can be sent there's a load of design choices about what is sent.

Is aliasing within the original preserved?

Is potential aliasing introduced?

Is it compressed in transit?

Is it delta encoded with respect to information that was sent previously?

Is there some initial blob of information every instance starts up with that messages are deduplicated against?

And that's before the behaviour on mutation - when one machine changes the data, how and when is that change reflected on other machines?

> All data is serialisable. I'd say that's axiomatic. Data is information with a concrete representation.

I Do not disagree. I think the problem comes from us using the term serializable to liberally.

> My best guess is there's some limitations caused by the stack under your implementation that you haven't worked around, but I don't know what those limitations are or which ones you plan to resolve.

Agree. Feedback received, and thanks. This will be worked in.

But I'll attempt to answer your questions here:

> Is aliasing within the original preserved?

No. The data being sent over the network is copied by value, not reference - if that makes sense.

> Is it compressed in transit?

Yes. With Message Pack.

> Is it delta encoded with respect to information that was sent previously?

I'm not sure I follow, but all calls are independent of each other.

> Is there some initial blob of information every instance starts up with that messages are deduplicated against?

> And that's before the behaviour on mutation - when one machine changes the data, how and when is that change reflected on other machines?

Not sure if this helps, but this [1] talks about the architecture. There's a C&C. The control-plane is the orchestrator for all machines.

[1] https://docs.differential.dev/advanced/architecture/