[jeffbee]

Google itself was shipping transformer-based features to consumers within 2 years, notably BERT for websearch.

[umeshunni]

and Google Translate: https://arxiv.org/abs/1804.09849