[solarwindy]

When framed like this, it's quite unsurprising that LLMs struggle to emulate reasoning through programming problems: there's just not that much signal out there. We tend to commit what already works, without showing much (if any) of the working.

A test for generality of intelligence, then: being able to apply abstract reasoning processes from a domain rich in signal to a novel domain.

Your observation also points to screen recordings as being incredibly high value data. Good luck persuading anyone already concerned for their job security to go along with that.