[xenadu02]

> Despite jokes about "better compilers", compilers are getting better

The compiler has to make static decisions. The hardware knows what is actually happening. There is an inherent information asymmetry at work that a "sufficiently smart" compiler seems unlikely to overcome.

My intuition says software can't beat the speed of a superscalar OOO CPU anymore than a GP CPU can beat a roughly equivalent DSP for algorithms suitable to run on the DSP, but I have no proof for that.

I'll also note that we've been promised "smarter compilers" for decades. Intel has tried that route several times. No one has ever made it work.

[dnautics]

> The compiler has to make static decisions.

Pretty sure I mentioned JITting in my comment.

> My intuition says software can't beat the speed of a superscalar OOO CPU anymore

How good is good enough? I mean we have distributed tensor flow which is basically on the fly compilation that can reorganize your computational graph around nodes with gpus separated by network latency, or Julia where you can drop in a GPUarray as a datatype and move computation to the GPU without changing your code.

If we go to something a bit more baroque, java is within 1.5 of c/c++ these days

Could you hand roll a better solution? Probably. Would it be worth it? Doubtful.

[repolfx]

I think it's definitely worth exploring this angle because modern JIT compilers have become very advanced, and there's still a lot of juice left to squeeze there. Look at some of the things Graal is doing and it looks a lot like what OOO speculation is doing - it'll recompile branches on the fly based on profiling information and things like that.

[gpderetta]

Nvidia Denver couples a software based jit/translator with an inorder VLIW backend. It is vulnerable to spectre.