[fat0wl]

This is a cool thought exercise I suppose, the idea of within a program throwing a particular function to an offsite parallel-compute engine. I just imagine it will complicate platform integration & bleed money given that it is in js (plus API headaches abound, as you mention)... If the same concept of dynamic provision/deploy/process/collect could all be done from a Java/C++ app I'd be very intrigued, but I suspect it is already near possible or achieved trivially through AWS APIs, albeit it probably with a manual deploy involved.

I think I remember this concept in Matlab back from when I did some research in grad school -- basically an instance of Matlab can be setup as a compute server, and the parallel processing functions of Matlab code on other computers can portion out work to it. This is the ideal model in my mind.

1. Write high-performance code in any language with some function that should be happening remotely in parallel.

2. Configure AWS to auto-provision the resources necessary.

3. Execute code, have it behave as if it is all running locally.

Really all these things can already be achieved with SOA, RMI, message queues, etc., the trick is just in making it transparent to the programmer so there is no deploy step involved. With the right spec it could even become platform agnostic (change small config file somewhere to target different cloud platform... would be nice to see a JSR about that in the near future!).

[nostrademons]

This is largely the architecture behind MapReduce or Hadoop Streaming. Write high-performance code in any language with an isolated parallelizable function, configure some cloud to auto-provision workers that run that function repeatedly on millions of records, execute code & pretend it's local.