This is true, and one of the reasons why I feel that we have come too far too fast in the hardware department, the degree to which Moore's law has enabled sloppy and wasteful programming should not be underestimated.
Is there any kind of index of software inefficiency? Something like "number of operations required by the software stack to display a sentence on screen", with a trend over time.
Input latency's one potential convenient proxy for this, once you adjust for display latency. "How long does it take the computer to do pretty much the most basic operation an interactive computer system performs?"
Considering an IBM 8088 did way better on that than modern computers, with like 1/5,000 of the processing power (figure ass-pulled but probably not far off), indicates to me that the trend is really damn bad.
This is very much application dependent, so you'd have to take a 1980's application, say Visicalc and extrapolate to today, say Office 365 or Google Sheets.
This you can then do for different application domains, and both in absolute terms (# cycles wasted) as well as relative terms (%age of cycles wasted).