[theturtletalks]

Is there a limit on how big the context size can be?

[PeterisP]

There is a soft limit due to the computation required; the currently used model architectures are quadratic with respect to context size, so if you want ten times larger context size, that's going to need a hundred times more effort.

[kickette]

There’s no theoretical limit