I'm sure Richard the Lionheart would've disagreed with you while locked up in Dürnstein castle.
The walls are only good if you're the one who controls access.