[mjhay]

There's also been some work on more general Lie-group equivariant transformer models.

http://proceedings.mlr.press/v139/hutchinson21a/hutchinson21...