Hacker News new | ask | show | jobs
by irq-1 2394 days ago
I was thinking about something similar to extfuse, but for a remote filesystem. Here's some note scraps:

> Server abilities can be changed by uploading JS functions/libs..

> Want to search a file formats meta-data on the server? Upload JS to read the meta-data and index it, and provide the search options..

> Servers should also use the best options for the tasks they provide: Grep to seach text -- and not Grep like functionality, but the optimized binary for the platform the server is running.

> Locking, linking, ACLs, and binary access should all be changable by JS.

1 comments

This is an interesting idea. Sort of the inverse of a web app. Part of the problem with the cloud for large datasets (ie genomics) is getting the computation close enough to the data (the UI being the third leg of the stool). If you could upload small processing scripts (or ebpf/wasm) to the exact node where the data lives in real time, it might open up some novel techniques.

Kind of like current serverless tech but instead of running at the edge, you run on the storage node.

EDIT: removed statement about network speed vs ssd speed. Pretty sure I was way off.

I believe this was a huge strength of MapReduce back in the day. The mapping and reducing of initial data would happen on the node the data was stores in. The only thing being transferred was the code and the results.