It's exactly the sort of thing people who didn't get a job say was the reason ("ah they didn't like me because I'm not from Harvard") and it's exactly the sort of thing interviewers don't actually say.
Maybe he thought it - he might have said something like "I don't think we're looking for someone with your background" or "we're looking for someone with different skills", but people always read what they think into comments like that.
There's almost no chance he actually said "Sorry we're looking for someone from Harvard" (especially as that is obvious from the CV so you wouldn't even get an interview).