[dismantlethesun]

I haven’t properly investigated their dataset they trained on this, but from the responses it gives I would bet that they dataset is skewed towards news articles, press releases, and infomercial or edutainment copy. It does really well when it can write a long impersonal article about something, but doesn’t seem good at the random sort of things GPT3 can do like autocompleting source code.

[datenhorst]

That makes a lot of sense. I played around with it in German and Italian and the results often times read like they were google translated from English. So there must be lots of auto-translated content from spam sites in the corpus, I guess.

[29athrowaway]

GPT3 can answer questions, in comparison this model works a little bit better than a markov chain.

http://joshmillard.com/garkov/