[tekacs]

Labs are trying to make long-horizon work. Even if you're a coding agent, adding more and more surface area is distracting to that goal. There is reason that RL over long traces should, at least in principle, optimize for building in ways that help the result fit in the model's context window.

A meaningful risk of course is that the tools available to the model (ripgrep + fancier semantic approaches) allow it to do a good job of reasoning over things much larger than its context window, and so it doesn't pay the penalty sufficiently to fix it.

[conartist6]

Does that not sound a little silly to you when you say it? Should I invest in becoming a memory athlete as a way of becoming a better software engineer? ...or should I learn how to build and use tools?

[gnatolf]

While I don't disagree, memory certainly was more of a restrictions on us humans than it is on llms. Therefore, the answer may not be as obvious as it seems. We build abstractions to reduce (memory) footprint of features, right?

[chongli]

Humans built codebases many millions of lines long, well before LLMs existed. Human memory has not been a restriction on us in a long time.

Look at all the libraries full of books we've built. It's useful for more than mere training sets.

[conrs]

I think the trick here is plural; I guarantee no single human knows all 1 million lines. Note this is different than knowing how to orient yourself in a million line codebase quickly.

The limit here I think the ancestor comments are getting at is cognitive load, which is real and measured. We only have so much memory to devote to a "stack" when executing, and it's usually quite constrained.

[chongli]

Note this is different than knowing how to orient yourself in a million line codebase quickly.

Hence my library mention. Humans have been doing this for millennia: orienting ourselves within a library (the physical kind, full of books) and calling upon its information resources as needed to accomplish tasks (research). Ultimately, it's all just one big cache hierarchy. Your short term memory, your long term memory, the book in your hands, the desk at the library, the nearby shelves, the card catalogue, the stacks, the inter-library loan system.

To manage it all, we humans have developed our abilities for abstraction. When we build clean, tight abstractions we reduce our cognitive load. Perhaps the best abstraction we've built so far is the TCP/IP and web stack. We don't need to care at all about the hardware details of a server in order to talk to it. It's such a powerful and airtight abstraction that we take it for granted.

I'd like to hear from more people who have spent a lot of time building with LLMs, because so far what people are saying is that these models do not have the ability to reason about and build the kind of marvellous abstractions us humans have built.

[conrs]

Fair, my mistake.

I've built a lot with LLM's, my experience sort of but not really tracks that. I've had to course correct a few bad abstractions but the larger the code base becomes the better it seems to be at reusing things. Maybe this is because of types, or spec-first development (with OpenAPI), or black box integration testing - but also maybe not. But generally I have to think about the abstractions and let the LLM fill in the details with rare exception.