[colechristensen]

These models have probably been trained on a large portion of the Internet.

The training data however can be a few years old.

I have noticed this when trying to get it to generate some code, it was clearly using considerably old versions.

[codebje]

> I have noticed this when trying to get it to generate some code, it was clearly using considerably old versions.

I have noticed the same thing when trying to use junior devs to generate some code, usually correlating with how old the most popular Stack Overflow question on the subject is.

[colechristensen]

This is how we decided to describe chatgpt’s programming ability the other day. A very fast but somewhat inadequate jr developer. Is amazing really but you also have to ask for the same thing a dozen times and still spend quite a while figuring out what’s wrong with it.

[Godel_unicode]

What I have been doing is asking it for code and then asking it to find bugs in that code. The bug finding works relatively well.