As someone who also does this, I find that the problem becomes not processing the information, but retaining it, and I haven't found a solution to that yet.
I doubt regular folks can reach 5x or even 2x with good retention. Perhaps if it's a podcast of people chatting about random things, it's possible, but any intellectually stimulating content you probably will have problems. Visually impaired people probably have an advantage on account of having their entire visual cortex freed up and possibly able to help with this (not sure if it does, neuroscience majors here?)
(As a sighted person) I watch all my videos or podcasts at 2x. Depending on how new or dense the content is, I might have to occasionally dial it back, but I think I do fairly well at 2x (usually, this is only the case if I'm distracted or the slides are going by too quickly as well). Often I can even up it to 3x.
Maybe retention is worse, but I'm still better off. Higher speeds are more engaging and enjoyable to me, as bad speakers and long pauses matter less. Therefore I benefit from watching more talks and lectures. For better retention it is preferable to occasionally re-watch good talks for spaced repetition.
The Video Speed Controller extension is indispensable part of my browsing experience, mostly for keyboard shortcuts to speed up/down and << >> videos.
I used to listen/watch everything at 1.5x-2x speed but found a similar result. Now I only increase the speed if it's information I only need to process once (e.g the news) but not remember later (lectures, tutorials etc).
I've been doing that lately (1.5x-2x), but taking notes, and making sure to work on any exercises provided after the lecture. I need notes for retention even at 1x, so it doesn't feel like extra work.