[JohnBooty]

    the optimal amount to spend on software optimization 
    is at least a substantial fraction of your hardware budget.

This has been a banging-head-against-wall sort of struggle every place I've worked on software, without AI even coming into the picture.

At one startup they were spending millions of dollars on AWS and complaining loudly to us about AWS spend and yet... god forbid the engineering team devote any resources to optimization passes instead of rolling out more poorly considered features, and hiring more engineers, because the existing engineers are struggling to be productive because everything is so unoptimized, and also because they have to spend a bunch of their time interviewing and training new hires.