I don't understand what you're doing. When you get an individual cert, are you adding another trusted authority to verify that cert? If you're just trusting the individual cert, you're exposed to MITM.
Yes, you're exposed to MITM. But if you permantly mark the cert as trusted, and the MITM goes away, you'll know somethings has changed. You'll be blind as to which way things changed, but at least you'll know to investigate.