I often do too! But I certainly also had some professors in college who were frankly just really bad at conveying their expertise in a way that was accessible to people who were not also already experts. Some SMEs happen to have good communication skills and can be really fun to talk to, but the systems that produce experts don't, in my experience, select for these attributes, so whether or not you get them is pretty luck-of-the-draw.