Hacker News new | ask | show | jobs
by anderspitman 2388 days ago
This is an interesting idea. Sort of the inverse of a web app. Part of the problem with the cloud for large datasets (ie genomics) is getting the computation close enough to the data (the UI being the third leg of the stool). If you could upload small processing scripts (or ebpf/wasm) to the exact node where the data lives in real time, it might open up some novel techniques.

Kind of like current serverless tech but instead of running at the edge, you run on the storage node.

EDIT: removed statement about network speed vs ssd speed. Pretty sure I was way off.

1 comments

I believe this was a huge strength of MapReduce back in the day. The mapping and reducing of initial data would happen on the node the data was stores in. The only thing being transferred was the code and the results.