[PartiallyTyped]

Language Models produce a high probability sequence of words given history (or an approximation of it). This is the only paradigm that we know works for language synthesis.

What the creators of this page did is turn that into its head, and use exactly that reasoning to identify candidate passages as computer generated, exactly because they have access to those probabilities, so it's not a viable approach to improving the language model directly.

With ChatGPT however, we have 2 models working , a language model, and a ranking model. The ranking model is trained to order the results of the language model to look better to humans. The suggested approach could be used to help fit the model by ranking lower probability sequences higher, but this comes at the cost of increased computation time by generating many more sequences, and constructing incoherent output.

[feoren]

> This is the only paradigm that we know works for language synthesis.

No, it's the easiest paradigm we know works for language synthesis. The other way to synthesize language is to understand what you're saying. This is "old-school" AI (we wouldn't even call it AI now), done with if statements, expert systems, and queries of a robust, structured data model. The bullshitting capabilities of neural networks have skyrocketed so far as to dwarf the "expert system" approach, but it's still there, slowly getting better, and still the right choice for many situations.

What I'm excited about is combining the capabilities of both. Right now there's a huge gap between the two.