There has to be a name for the fallacy where people in our profession imagine that everything in technology follows Moore's law -- even when it doesn't.
We're standing here with a kind of survivorship bias because of all the technologies we use daily that did cost reduce and make it. Plenty did not. We just forget about them.
He isn't saying they won't ever come down, he's saying they will not be coming down any time soon due to structural factors he discusses in the article.
Computers do get more powerful, but the price for a decent system has been about the same for a long time. $1500 got me a good system 10 years ago, but I'd still be paying $1500 for a good system today.
The first mobile phone, the Motorola DynaTAC 8000X, was launched in 1984 for $3,995 (more than $12k in 2025 dollars). So we should expect a 12x cost reduction in LLMs over 40 years.
IBM 3380 Model K introduced in 1987, has 7.5 GB of storage and costed about $160000 to $170000, or adjusted for inflation it is $455000 in 2025 US dollars, that's $60666/GB. A Solidigm D5-P5336 drive that can store 128 TB costs about $16500 in 2025 US dollars, that is $0.129/GB. That's a 470279x price reduction in slightly less than 40 years. So what is likely going to happen to LLM pricing? No one knows and both your example as well as mine doesn't mean anything.
We're standing here with a kind of survivorship bias because of all the technologies we use daily that did cost reduce and make it. Plenty did not. We just forget about them.