[jackblemming]

The state of art neural net architecture, whether that be transformers or the like, trained on self play to optimize non-differentiable but highly efficient architectures is the way.

[hackerlight]

According to Hinton, before transformers were shown to work well, learning model architectures was Google's main focus