It’s now moving faster than ever. Huge strides have been made in interpretability, multi modality, and especially the theoretical understanding of how training interacts with high dimensional spaces. E.g.: https://transformer-circuits.pub/2022/toy_model/index.html
Thanks, this seems interesting. I'll give it a read. I admittedly don't keep tabs as much as I should these days. I feel like every piece of AI news is about LLMs. I suppose I should know other people are still doing interesting things :)