[lanstin]

Math professors are remarkably good at answering math questions or solving ad hoc math puzzles correctly. Also at stepping back and asking, why are you trying to show this result, what do you think it has to do with the thing we are trying to prove. Much better at knowing when the student has gone off the path in some subtle way than the LLM (without a Prof Tao in the chat).

Professor Tao also put out some YouTubes of him working with an older LLM to do Lean proofs, and his intelligence matters - things where I would be stuck for hours trying to understand what was failing in the model proof were just instantly clear to him and fixed in thirty seconds.

And I am an ok coder (rather than a bad maths grad student), but the LLM will happily thrash around the edges of a problem with me with no clear convergence when I don’t have that clear insight and the problem is weirdly presented enough; I still find the trick to walk around the block and disengage and then return knowing exactly what to do (now prompting it to the right thing) to be a super power for getting what I want out of the coding system.