So in 50 years the equivalent of a gpt4 training cluster from today's datacenters will fit in a cheap cable, and it will run over 100 times faster than a full cluster today.
I'd say even more impressive is that, given an ARM Cortex X4 runs at almost 100x the clock speed, up to 14 cores, has caches, branch prediction, floating point, specialized instructions, gpu, etc., it's probably 100000 times faster than the CYPD4225. So given GPT4 training took ~100 days, it means a typical cell phone equivalent in 50 years will be able to train a GPT4 equivalent from scratch in under a second.