What about I work to publish crowd-sourced programs and measurement scripts: so that anyone can make their own comparisons, on their own hardware, against whatever other language implementations they want to write programs for :-)
The only thing I wonder about is allowing any given set of hardware to be compared with another set of hardware with anything approaching accuracy.
What would be awesome is if someone could figure out whether they'd need Raspberry Pi Zero to run some algorithm or if they could get away with just using a BBC Micro:Bit, by looking at a benchmark run submitted to the site from an i9-7940X.
Sadly I suspect the amount of realtime introspection needed (memory speed, practical cache coherency (per level), bus saturation, instructions used per cycle) would make this difficult - particularly because, even if a given benchmark was going to go fishing for performance counter info (and I just learned that even the Gen1's ARM 1176-based PMU provides a few, including one that counts instructions), benchmarking memory I/O is a bit harder; PCI DMA MMIO debug cards only map a small range of memory, like 256k or so, and I don't know if such cards can back the region with actual RAM. I suspect the access latency would be so different than from normal RAM that even if this did exist it'd never be used. Sigh.
And then differences in compiler optimization approach would have to be taken into account, and thorough understanding of assembly language for all target architecture(s) would be needed to have the time of day to page through the diff analysis...
Hmm, I think "simple" got left behind a few thousand miles ago. It's kinda (morbidly?) fascinating how different that different architectures are on the ground, hah.
- I now realize (after having actually had a proper look) that while the Micro:Bit does use "JavaScript" and "Python", they're different, microcontroller-specific implementations, and I should have used a different example there, like an Intel NUC or perhaps an ODROID board or similar.
The only thing I wonder about is allowing any given set of hardware to be compared with another set of hardware with anything approaching accuracy.
What would be awesome is if someone could figure out whether they'd need Raspberry Pi Zero to run some algorithm or if they could get away with just using a BBC Micro:Bit, by looking at a benchmark run submitted to the site from an i9-7940X.
Sadly I suspect the amount of realtime introspection needed (memory speed, practical cache coherency (per level), bus saturation, instructions used per cycle) would make this difficult - particularly because, even if a given benchmark was going to go fishing for performance counter info (and I just learned that even the Gen1's ARM 1176-based PMU provides a few, including one that counts instructions), benchmarking memory I/O is a bit harder; PCI DMA MMIO debug cards only map a small range of memory, like 256k or so, and I don't know if such cards can back the region with actual RAM. I suspect the access latency would be so different than from normal RAM that even if this did exist it'd never be used. Sigh.
And then differences in compiler optimization approach would have to be taken into account, and thorough understanding of assembly language for all target architecture(s) would be needed to have the time of day to page through the diff analysis...
Hmm, I think "simple" got left behind a few thousand miles ago. It's kinda (morbidly?) fascinating how different that different architectures are on the ground, hah.