No. Blame browsers for not being user-agents any more. Browsers can cache websites, but only if websites tell them to do so. Users are never asked ( there is no setting ). And since websites usually don't want to be cached (for too long) browsers have to reload them ( which takes time ). Thats the reason Back-buttons are slow. Thats ( one of ) the reason(s) everybody has 100+ tabs open.
SPAs and infinite scroll are (maybe) bad, but not the cause of this problem.
Some infinite scroll-type pages consume ridiculous amounts of memory the further you scroll (and if the page disallows direct navigation to a point in history and you need to find something older that you didn't save a direct link for, you've got no choice but to scroll, and scroll, and scroll…), so limitless caching isn't quite the right answer, either…
SPAs and infinite scroll are (maybe) bad, but not the cause of this problem.