[coldtea]

>their mistakes aren't based on their understanding, it's basically random guesses.

Not random across their whole training set. Random across related concepts bundled together in the training set. Which is not that dissimilar to human mistakes.

A human's mistakes are also based on going from one option in their training and not another, where the two are close together but one is not appropriate and doesn't fully cover the expected result.

That's obvious in a typo (you get close to the target word but miss it just so), but also in off by one errors (you're still in the proximity of the correct loop you should have written), all the way to picking the wrong architecture or patter n (you still chose among patterns for the worse fit you've picked, you don't suddenly start using cooking recipes).