Yeah. When I’m interviewing folks at my current company, and the Q&A portion begins, I tell them they can ask whatever they want or just reclaim the time; my notes are sealed. More interviewers should operate like that.
You can tell them that, but I don't think I'd believe it as a candidate. I'll assume that every interaction I have with you up until an offer is extended is going to influence your perception of me, whether you're taking notes or not.