[moozilla]

My guess is the missing ingredient is the back and forth communication with other humans. Surely if you quantified the amount of all of the information (verbal and over other channels) that is traded back and forth by humanity in a small timeframe it would dwarf the current training set. I think is the idea behind ChatGPT - your conversations with it are the next set of training data. I can see the argument that this sort of conversational data is not as valuable as say scientific journal articles, but maybe the volume makes up for that?