| That seems daft. You can, I suppose, contract your code so that it’s context free and uses less tokens, but that makes it more confusing for humans and language models. Taken to the extreme, you can see obviously with one letter functions and variables like i, j, k the model will be able to infer literally nothing and, thus, produce arbitrary nonsense. Clearly the solution is to do what we already do to manage complexity which is to decompose large tasks into smaller black box modules with an api where the (large number of tokens) implementation is hidden and not known or relevant to using it. If you give an LLM a function signature and good description, maybe some usage examples, it doesn’t need the implementation to use it. Terseness decreases the ability of LLMs to process code; it doesn’t solve context length, and even at best it doesn’t scale. 100k tokens is plenty. You don’t need to do anything like that. |