[willy_k]

It took 6 years to get from Attention Is All You Need to a consumer product. Up until ChatGPT’s release, LLMs were obscure and really were just fancier autocomplete. Deepmind was largely a speculative research division until Google decided to play catch up, and even that took a bit of time as they figured out how to proceed in a way that wouldn’t cannabalize search. So in short they didn’t realize the potential. Thats my somewhat naive take.

[jeffbee]

Google itself was shipping transformer-based features to consumers within 2 years, notably BERT for websearch.

[umeshunni]

and Google Translate: https://arxiv.org/abs/1804.09849