Are you sure the video was registered as seen ? Sometimes on my phone I watch a video and I don't see it in my latest seen videos. It's an issue with Play Service I think.
I constantly have the same issue. Youtube even shows the video as already watched. I click not interested, then already seen, but another one pops up in it's place. Quite annoying.