> To me it absolutely is, because you can only keep borrowing money and paying the (increasingly large) interest on it for so long.
No. This is modelled and understood reasonably well. There are limits, of course (though the Reinhard-Rogoff paper stipulating a significant drop in growth beyond a certain level of debt to GDP is considered flawed [1]). But moderately growing debt can be sustained indefinitely.
A blog post explaining this well would be useful indeed, I wonder whether Krugman has written one (but haven't found one after a quick search).