|
|
|
|
|
by BoiledCabbage
1123 days ago
|
|
I think you misunderstood the token limit. LLMs don't block your buffer, they simply take the final n tokens of all of the input you've shared. Plenty of users have seen this. It will still function, but anything you previously shared and referenced above it will lose context on. And if you ask it about that earlier content, it will do its best to hallucinate a reasonable answer of what might have been in your buffer before the cutoff. Separately you may have found a physical hard limit with a bug that crashes the system, but that's not what's meant by a token limit in LLMs. It's a limitation of the architecture itself of any LLM. |
|