[kragen]

You seem to be agreeing with the story's thesis, rather than disagreeing. The story claims that we get an enormous amount of data from which we could compute much more than we do. You, too, are claiming that we get an enormous amount of data from which we could compute much more than we do. If that's true, then we aren't limited by our data, which is what I meant by "data-limited"—although you seem to mean the opposite, "we get data slower than we can analyze and process it", in which we are limited not by the data but by the processing. This tends to rebut the claim above, "If you had AGI tomorrow and asked it to cure cancer, it would just ask for more experimental data and resources."

It may very well be true that you could cure cancer even faster or more cheaply with more experimental data, but that's irrelevant to the claim that more experimental data is necessary.

It may also be the case that there's no "shortcut" to simulating a human body well enough to test drugs against a simulated tumor faster than real time—that is, that you need to have enough memory to track every simulated atom. (The success of AlphaFold suggests that this is not the case, as does the ability of humans to survive things like electric shocks, but let's be conservative.) But a human body only contains on the order of 10²⁴ atoms, so you can just build a computer with 10²⁸ words of memory, and processing power to match. It might be millions of times larger than a human body, but that's okay; there's plenty of mass out there to turn into computronium. It doesn't make it physically unrealizable.

Relatedly, you may be interested in seeing Mr. Rogers confronting the paperclip maximizer: https://www.youtube.com/watch?v=T-zJ1spML5c