[chaitjo]

I am guessing he means that Transformers 'discover' parse trees in sentences, as they operate on fully-connected graphs but end up focusing on the most important connections through attention.