[Jensson]

Well, my point is that you perceive progress to be fast since you went from not understanding what existed to later getting in on it. That doesn't mean progress was that fast, it means that you just discovered a new domain.

Trying to extrapolate actual progress is bad in itself, but trying to extrapolate your perceived progress is even worse.

[james-revisoai]

Yeah you have hit the nail on the head here. A lot was predictable with seeing that GPT-2 could reasonably stay within language and generate early coherent structures, that coming at the same time as instructions with the T5 stuff and the widespread use of embeddings from BERT told us this direction was likely, it's just for many people this came to awareness in 2021/22 rather than the 2018-2020 ramp up the field/hobbyists experienced.