[naikrovek]

we have always been memory-bound, in one way or another, even today.

the difference in performance between an application using RAM with random access patterns and an application using RAM sequentially is far more than you are expecting it to be if you haven’t actually measured it. an order of magnitude or more for sequential stuff over random access. having your data already in the L1 cache before you need it is worth the effort it takes to make that happen.

[kaba0]

Indeed, but in the case of your average application it is not only lack of will/expertise to optimize, but also simply that the program domain has a much more random memory allocation pattern. Most programs are not operating in a single hot loop on terabytes of data.