I have seen HSMs in a concrete fireproof room, inside another concrete room, inside a regular room, inside a nondescript building. This one required the use of 3 persons (one to authorize entry past the 2 guys with rifles, one to unlock the vault rack with the hsm, another to authenticate to the HSM to perform work).