[emerongi]

The whole connection hangs - it's not the server or buffering and I doubt it's the ISP.

Reading more about it, you are correct about 1min being too long, therefore it's probably not (just) bufferbloat.