How do you know the fab you send your design to doesn't implement a backdoor?
It really wouldn't be that hard to compare what you received from the foundry with the masks you sent them. There are services that decap ICs and attempt to reverse engineer them. Here's the first hit I found on Google: http://www.intelligservices.com/Services.htm
You wouldn't need the reverse engineering. All you would need to do is to compare that all the metal layers (and perhaps poly) match what you sent. That's an almost trivial comparison. The hard part of reverse engineering is in figuring out what the circuitry actually does, and you already know that, since it's your chip!
It could be possible for a fab to alter diffusion layers to change the functionality of a chip. That would be harder to detect by services such as the one I mentioned. But it would be very hard, very time consuming, to attempt to hack in a backdoor by only messing with base layers, rather than messing with metal or with poly (where changes are easily observed).
If there were to be a backdoor anywhere in a design, it would be in IP you used on your chip, that you got from either your foundry or from a 3rd party IP supplier. It would be easy enough to hide all kinds of stuff there.
That is exactly my point. If you don't trust Intel, then who are you suggesting should build your processor? Unless you are going to design and fabricate your entire system top to bottom, and have no other persons involved who could surreptitiously insert evil code, then you're going to have to accept some degree of trust in a 3rd party.
It's not just Intel you're trusting, it's the OEMs, who also have an EXTREMELY poor record. Just take a look at the ENORMOUS number up BMC and auto-update vulnerabilities. They really just either couldn't care less, or are deliberately making machines vulnerable. Things are being pushed into IME, that aren't optional, often aren't wanted by users (or snuck in there so they wouldn't know), don't all have to be there, can't be verifiably disabled or overriden by end users. Even if it weren't backdoored (unlikely), it presents an ENORMOUS attack surface at the very worst possible privilege level. Open source designs that are at least inspectable by researchers would be a start. But more importantly - allowing users to CHOOSE whether they want to override software (it IS after all software and not hardware). Why shouldn't we be permitted to run Libreboot etc on modern hardware and know that when we turn a machine off, that it's off, without unplugging network, power cables, batteries, etc?