Ok, NOW imagine what 10,000 3.84GHz cores would cost (100 times more aggregate cycles per second than 1x 384GHz core). What's that, you figure a measly $10-30M instead of more money than exists on earth?
Any research simulation is going to want to be parallelized anyway. You'll bump into the limits of the 384GHz core, no doubt about that, at which point you are back to distributed computing. For limitless complexity and limitless appetite for computing power, distributed computing will always be the answer.
I'm afraid the machine you'd get would be a furnace or a torch... AFAIK power that you'd need to dissipate is linear on frequency, so unless we will discover technology to radically lower the voltage used in CPUs or some other parameter, dissipating 100x heat would be kind of a big problem for such CPU.
In one clock cycle, light would be able to travel about 780 micrometers. The real propagation speed of signals on a chip will be substantially less than the speed of light, and it gets slower at smaller feature sizes. This could be tricky.
Unfortunately, many machine learning algorithms aren't easily parallelizable. Consider any boosting ensemble learner, such as the popular random forest, which optimizes the result from the previous iteration. (I run the machine learning startup diffbot.com and we could really use faster cycles.)
The machine learning I did was never limited by CPU power, but rather my algorithm and especially the states. Asking for better performance without better states is garbage-in-glory-out.
(hint: probably more than the world GDP. Each)
Ok, NOW imagine what 10,000 3.84GHz cores would cost (100 times more aggregate cycles per second than 1x 384GHz core). What's that, you figure a measly $10-30M instead of more money than exists on earth?
Any research simulation is going to want to be parallelized anyway. You'll bump into the limits of the 384GHz core, no doubt about that, at which point you are back to distributed computing. For limitless complexity and limitless appetite for computing power, distributed computing will always be the answer.