[bra-ket]

does it mean we don't need gradient descent after all to achieve the same result?

[sanxiyn]

Nope, Transformer is still trained with gradient descent.