[auntienomen]

I found Cosma Shalizi's notes on the subject pretty insightful.

http://bactra.org/notebooks/nn-attention-and-transformers.ht...

Definitely read through to the last section.

[carterschonwald]

Thanks for sharing! I find cosma’s writing enlightening always. And read his stuff far less than I should