[RandyRanderson]

Also, we need to show that the data costs, dev costs, training costs, maint, etc, and inference time costs are cheaper than human alternatives for some reasonable loss function, for most tasks.

If you look at the FN and FP rates for many tasks, the SoTA Transformer models are all VERY high vs humans and inference times are maybe only 10x.

At 100x or 1000x the transistors, data, etc and even at 1/10 the loss, ML solutions are likely not competitive for many tasks.

[sanxiyn]

The article is about we have no fucking idea how things like 1/10 loss translate to capability and that's why we can't make statements like "even at 1/10 loss ML solutions are likely not competitive for many tasks".