[nerdponx]

You're entitled to your own opinion of course, but your conclusions appear to be based on beginner-level misunderstandings. That doesn't seem like a constructive or productive way to conduct oneself through life.

[h2odragon]

you should see my rants about why normalizing weights is a bad idea and how a limited context window is effectively random interpolation