This is why you need defense in depth: physical seals on components, either protecting cables or keeping them clearly visible where someone can notice tampering, and – above all – having the physical space setup to strictly limit someone's ability to bring arbitrary objects in or spend time alone with sensitive hardware.
Consider what someone with the time, skill, and access to do that could also do without that: opening the case and directly installing some sort of device, planting a camera which records you typing passwords in (“oops, left my cellphone sitting out. Won't happen again!”), planting a radio receiver which opens up all sorts of side channel attacks, installing a passive network tap, etc.
A guard with a metal detector and strict limits on what you can bring into the building or what tools you can use inside is going to do a better job preventing all of those.
Consider what someone with the time, skill, and access to do that could also do without that: opening the case and directly installing some sort of device, planting a camera which records you typing passwords in (“oops, left my cellphone sitting out. Won't happen again!”), planting a radio receiver which opens up all sorts of side channel attacks, installing a passive network tap, etc.
A guard with a metal detector and strict limits on what you can bring into the building or what tools you can use inside is going to do a better job preventing all of those.