[lexpar]

Yeah, I guess this is fine if by "Transformers are Graph Neural Networks" we mean Transformers < GNN, rather than Tranformers == GNN.

"Sentences are a fully connected graph". Ok fine, but that's a graph with basically no information embedded in its structure. GNNs are supposed to be useful for graphs that have interesting structure, right?

[chaitjo]

Not necessarily. In fact, applying the Transformer/GNN on a full graph is seen by some people as 'discovering' or identifying some useful underlying graph structure. We had an interesting discussion on this on Twitter: https://twitter.com/PetarV_93/status/1233820829852618754