[zdragnar]

Invert that phrase. "__company__ isn't a good fit for me". Be clear that you are leaving on your terms.

The interviewer may still tell people that you gave up halfway through, but if anyone else is listening or the interviewer is accidentally honest, it is more likely to trigger change by emphasizing that the company is losing you rather than you are losing the company.

[mrozbarry]

I think that's a valid approach, too. It might just depend on the interviewee which is better. The inverted phrase is a bit more direct and dominant (which I don't think is bad), so I don't think it fits everyone's personality. That said, my recommendation is simply to be polite and leave, and do it on your own terms.