I agree it's ridiculous (though not in the way you meant), but he is telling the truth. I don't doubt that there are facilities which are watched as closely as you describe, but not all are. I have worked in a SCIF where the only thing stopping me from bringing in a phone was the fact that I said I wouldn't, and I keep my word.
I added “most of the time” to match my experience. If the entire building is a SCIF, I’ve usually left my phone in the car. If it’s just a room in a building, they usually have lockers outside and use the honor system.