[noashavit]

An architecture where compute grows linearly with context length seems dangerous. It can get very expensive as context grows and performance degrades