[lelag]

Since a single inference is limited by context length, a multiple agents model is able to process more context at each steps of the reasoning chain, which might improve the overall quality. However, given how easy it is getting to fine tune models, it's likely that multi-agent models will make a lot of sense to split the workload and assign each part to a specialized agent.

[wokwokwok]

> a single inference is limited by context length,

Yes.

> multiple agents model is able to process more context at each steps of the reasoning chain

What?

How can a multi agent model have more context at a single step? The single step runs on a single agent. It would literally the same as a single agent?

The multi agent approach is simply packaging up different “personas” for single steps; and yes, it is entirely reasonable to assume that given N configurations for an agent (different props, different temp, different models even) you would see emergent behaviour that a single agent wouldn’t.

For example, you might have a “creative agent” to scaffold something and a “conservative” agent to fix syntax errors.

…but what are you talking about with different context sizes? I think you’re mixing domain terms; context is the input to an LLM. I don’t know what you’re referring to, but multi agent setups make absolutely no difference to the context size.

[a_wild_dandan]

Their comment uses two (valid) context lengths: "organizational total" and "single agent." The latter is a subset of the former.

By analogy: no agent can summarize War and Peace, but several agents can, Peace-wise (sorry). Like AI map reduce. The question is thus "why not use one agent for this recursive merger?" Answers maybe being:

1. Different scholars (Russian lit. agents, ...war strategists?, etc) pay attention to different things with valuable insights

2. Multiple readers parallelize well, and some are faster than others

3. Managers can direct talent to (re)read chapters most relevant to their specialties, and coordinate meta-learning and communication

You might not get much mileage out of this approach with book summaries, but other domains are a different story (sorry).

[wokwokwok]

I’m not sure what this means.

Are you agreeing or disagreeing?

Yes, multiple agents with different personas will give different takes and may lead to emergent behaviour, eg. discussing the book.

Yes, they could run in parallel.

No, any single multi agent step will not have any more context than any other single step.

If you believe that the Nth prompt in a chat to a LLM eternal multiple agents has “more context” than a chat between a single agent (and itself) you don’t understand how this works.

…or you are choosing to invent your own definition of “context”.

[ca_tech]

I think this is right inline with the utility of multi agent models. Whether distributing tasks to specialized agents trained on domain knowledge or collaborating with context aware agents. I think the context is where we are going to find limitations early on especially when models are expected to work on live data. Rather than constantly retraining a model, you leverage a model that is already primed through in-context learning based on previous interactions and relevant data.

[ProofHouse]

context window is fast becoming a non issue (memgpt, SPR, sink tokens etc)