[larodi]

Google published a lot. I still wonder this original transformer paper, and the attention, etc. Why would they allow it to let go in the open? Perhaps because was intended for translation first before someone decided to loop it over itself? Or was so obscure to fellow researchers what do they actually publish?

[willy_k]

It took 6 years to get from Attention Is All You Need to a consumer product. Up until ChatGPT’s release, LLMs were obscure and really were just fancier autocomplete. Deepmind was largely a speculative research division until Google decided to play catch up, and even that took a bit of time as they figured out how to proceed in a way that wouldn’t cannabalize search. So in short they didn’t realize the potential. Thats my somewhat naive take.

[jeffbee]

Google itself was shipping transformer-based features to consumers within 2 years, notably BERT for websearch.

[umeshunni]

and Google Translate: https://arxiv.org/abs/1804.09849

[leoh]

Kind of makes you wonder if there are other papers from Google that are potentially also profoundly groundbreaking but overlooked

[whiplash451]

Groundbreaking includes the “explain to the world why it matters” part