Use a common prefix or suffix that you don't write down so that even if someone reads your little book (which is unlikely anyway), they won't be able to use the passwords.
Sure, this is great if your biggest concern is password security vs. password breaches. No one using pen & paper is creating new passwords for every site they register for, so they're probably just reusing the same few passwords (or, maybe, with small variations) between sites. Which means they're constantly putting themselves in a position to be hacked, regardless, since breaches expose all your other accounts when you have just one email address.
My parents tried that, but they kept misplacing the notebook in which these passwords were kept, and the passwords were too simple and guessable by half in any case.