[gptgpp]

Except it should have been trained on probably tens/hundreds of thousands of 2016 advent of code solutions posted on github and other sites, shouldn't it?

It often starts hallucinating the input in the exact format advent of code gives, so I'm like 99.9% certain it has been trained on a large corpus of advent of code.

Personally I use codex sometimes for debugging help so I agree it can do cool stuff. I just disagree it is "almost" right at solving problems -- it almost never generates code that even compiles for me when prompted to do anything not super trivial like advent of code brain teasers.

[rcme]

What is in the training data and what the model was optimized to do are two totally different things. And even then, tens of thousands of solutions may not be enough to train the model for this specific problem domain.