|
|
|
|
|
|
by cma
16 days ago
|
|
|
They tested them on formal languages of different power, and saw where they could generalize beyond the training data. Transformers failed to generalize pretty early on at stack machines/brace matching. It was a good bit older of a paper though, if I remember it's somewhat expected from the pure feed forward nature of them and limited circuit depth, where LSTMs have some recurrence. |
|
|