In college, I talked to a Japanese friend about it (I was studying Japanese at the time, so differences between the languages were a natural topic). He could tell the difference between "right" and "light" when I enunciated them clearly, one after the other to provide an immediate contrast between them, but couldn't otherwise. In most cases, context was enough to disambiguate the meaning.
So, he can (of course) physiologically hear the difference, but he wasn't conditioned to listen for it. There are features like that for me in other languages (tones in Chinese, as one glaring example).
You met a very small sample. The stereotype in the West is that Japanese can't do the L sound when it's actually specific versions of the R sound that give people not exposed to it early enough alot of issues
So, he can (of course) physiologically hear the difference, but he wasn't conditioned to listen for it. There are features like that for me in other languages (tones in Chinese, as one glaring example).