[yinser]

To add another anecdote to your question: the transformer became a part of the first context aware embedding model GPT-1. Not to say it couldn’t be done with another tool but it was first done with a transformer. Previous embedding models like word2vec, GloVe and fasttext were not contextually embedding and didn’t give you a language graph that would then go on to support a language model capable of “understanding” what you were saying or asking for.