[panflute]

True, yet they have a Moore's Law like growth going for properties like their context windows.. I think the larger problem with letting them be verbose is Occam's razor. The more verbose they are the more variant behavior they will have where any variation that is not strictly necessary is likely to include incorrect behavior.

[pona-a]

Attention is an O(n^2) algorithm. Combined with Moore's doubling, it will at best produce linear growth (assuming Moore's law is still remotely close to alive)

[panflute]

I don't think many developers like software bloat or what it has meant for our professional reputation but we would be dishonest if we predicted a future without outcomes where ugly brute force wins given all the constraints. It is not Moore's law that dictates context window that is only an analogy and so far it has been exponential growth that went from well bellow humans to more than a human can deal with in terms of short term tasks.