[darkteflon]

You mean manually pre-baking a DAG from the user query, then “spawning” other LLMs to resolve each node and pass their input up the graph? This is the approach we take too. It seems to be a sufficiently performant approach that is - intuitively - generically useful regardless of ontology / domain, but would love to hear others’ experiences.

It would be nice to know if this is sort of how OpenAI’s native “file_search” retriever works - that’s certainly the suggestion in some of the documentation but it hasn’t, to my knowledge, been confirmed.

[TZubiri]

No. The DAG should be "manually pre-baked" ( defined at compile/design time).

In runtime you only parse the "user question" (user prompt) into a starting and end node, which is equivalent to a function call.

So the question

"What league does Messi play in?"

Is parsed by the llm as

League("Messi")

So if your dag only contains the functions team(player) and league(team), you can still solve the question.

But the llm isn't tasked with resolving the dag, that's code, let the llm chill and do what it's good at, don't make it code a for loop for you

[darkteflon]

That’s very interesting. Does designing the DAG in advance imply that you have to make a new one for each particular subset of end-user questions you might receive? Or is your problem space such that you can design it once and have it be useful for everything you’re interested in?

My choice of words was poor: by “pre-baking”, I just meant: generated dynamically at runtime from the user’s query, _before_ you then set about answering that query. The nature of our problem space is such that we wouldn’t be able to design DAG in advance of runtime and have it be useful everywhere.

The answering process itself is then handled by deterministically (in code) resolving the dependencies of the DAG in the correct order, where each node might then involve a discrete LLM call (with function) depending on the purpose. Once resolved, a node’s output is passed to the next tier of the DAG with framing context.

[TZubiri]

You don't make a DAG for each question category. This is classic OOP, OG, Kay's version, you design subject-experts (objects) with autonomy and independency, they are just helpful in general. Each function/method, regardless of the Object/Expert, is an edge in the graph. A user question is simply a pair of vertices, call them I and O, and the execution/solution is a path between the two points, namely the input and the output.

The functions are traditional software (Code, API, SQL) the job of the LLM is only to:

1- Map each type of question into a subsystem/codepath. The functional parsing solution is the most advanced. But a simple version involves asking LLM to classify a question into an enum.

2- To parse the parameters as a list of key/value tuples.

The end. Don't ask the LLM to cook your food, clean your clothes or suck your dick. LLM is revolutionary at language, let it do language tasks.

We are not consumers of a helpful AI assistant, we are designers of it.

[Scipio_Afri]

Thanks for your perspective! Can you explain why oral is a bad idea?

[TZubiri]

IO STDs

[darkteflon]

Very interesting perspective - thanks for your time!