|
|
|
|
|
by actionfromafar
1302 days ago
|
|
In a simple Lisp machine, such as something resembling PicoLisp, I can't see why not - iff instead of car and cdr being just linear addresses in a memory, have them itself be hashes. Since everything is made up from car and cdr, it's easy going from there. There is no difference between running locally, or anywhere. Just look up the data by request or gossip, as you said. (For performance reasons, one might want to let a cons which represents "source" smaller than the size of a hash, be a literal representation instead of hashed. No need to do a lookup from a hash when you can have the code/data in the cons itself. Analoguous, don't zip a file when the resulting zip would be larger.) |
|
Lets say you wrote a imaginary program to sum a column in a csv:
(defun my-program () (let* ((raw-data) (s3-load-file "htpps://...")) ((parsed) (csv-parse raw-data)) ((column1) (csv-column parsed 1)) (mean column1))
In this pretend program we are using some 3rd party s3 library to fetch some data, some other 3rd party CSV library to extract the data. Now I want to run this on some other remote node. I know I can just sent that sexp to the remote node and have it eval it, but that is only going to work if the right versions of the S3 and CSV library are already in that runtime.
I want to be able to write a function like:
(defun remote-run (prog) (....))
That can take ANY program and ship it off to some remote node and have that remote node calculate the result and ship the answer back. I don't know of some way in lisp you could ask the runtime to give you a program which includes your calculation and the s3 functions and the csv functions.
In unison, I can just say `(seialize my-program)` and get back a byte array which represents exactly the functions we need to evaluate this closure. When the remote site tries to load those bytes into their runtime, it will either succeed or fail with "more information needed" with a list of additional hashes we need code for, and the two runtimes can recurse through this until the remote side has everything needed to load that value into the runtime so that it can be evaluated to a result.
Then, of course, this opens us up to the ability for us to be smart in places and say "before you try running this, check the cache to see if anyone ran this closure recently and already knows the answer"