Existing power stations that are paid off have an advantage considering cost of capital (nuclear and pumped hydro), but that advantage eventually theta decays when maintenance costs exceed new build costs. France is experiencing this now with its aging nuclear fleet [1]. Running the math on Tesla's calculator [2], a quarter worth of annual production (10GWh) runs $3,798,849,522 (~$3.8B). That includes installation (per Tesla). Can new pumped hydro be built cheaper than that? For sure, keep existing low carbon generation or storage in service as long as commercially reasonable.
For load balancing, these days, there’s really no need for giant monolithic storage projects. Every renewables project can — and should be made to — have storage built in to it. Companies leave it out as it would make there price per MWh higher, and, for some reason, governments let them bid without it.
(Add storage to offshore wind and it suddenly looks similar in cost to nuclear.)
So, no one’s suggesting replacing pump storage with giant battery banks, just spreading batteries and other storage across the grid.
We still have no technology capable of storing TWhs though, and that’s what we need to remove carbon from our power grid.
Dinorwig cost 0.5 billion GBP to produce in 1974 (roughly 1 billion pounds in today's money) and took 10 years to build.
Nobody is arguing Dinorwig should be torn down. However, it took 10 years to build dinorwig. It takes less than a year to build a battery version of Dinorwig with basically 0 geographic requirements. The cost would be roughly comparable for a battery backup (at $200USD/kWh, it'd be $1.8 billion to build a similarly sized plant. or 1.4 billion GBP)
[1] https://www.world-nuclear-news.org/Articles/EDF-revises-up-c...
[2] https://www.tesla.com/megapack/design