[kelnos]

That problem is solved through humility and social graces.

"Hold on a sec, I just wanna write down what I was doing so I don't lose my place."

If you want to add more, after writing it down, you can follow up with, "sorry to keep you waiting, but when I get interrupted I often forget some useful details about what I was working on, and I want to get that down before it leaves my head".

(The "when I get interrupted" is a bit of a cheeky way to point out "you are imposing on me, so the least you can do is allow me this".)