[Reelin]

> I've always wondered if its possible to write software that will run on computers with arbitrary number of memory heirarchy

I think that's already possible today - the Legion programming system claims to do exactly that (among other things). I have absolutely no experience with it though so I might be completely wrong.

(https://legion.stanford.edu/)

> By making the Legion programming system aware of the structure of program data, it can automate many of the tedious tasks programmers currently face, including correctly extracting task- and data-level parallelism and moving data around complex memory hierarchies.