I guess if you worry about it you could always install the fireman's key box you see on commercial buildings.