[mrkramer]

I had a similar idea but what happens when you stumble upon code, equations, tables, graphs etc.? Can LLM understand that as well?

For example; you are listening to the paper with some text2speech model and then it stumbles open code snippet or table or graph....what should happen next? Should model skip it or prompt you to look at the graph or table or whatever. Or should you write some software that tries to interpret graphs and other non-text content.