[tokioyoyo]

Fair enough. In my imagination, I can see people writing AI-first framework/architectures and a general trend for people to “migrate to such frameworks”, just like the push towards the microservices architectures in 2010s. A part of these frameworks would be “re-constructibility” by changing contracts in parts where it matters, and somehow the framework would make it easy for the LLM to discover such “parts”.

Honestly, i’m making stuff up, as I don’t think it’s feasible right now because of the context sizes. But given how fast things develop, maybe in a couple of years things might change.

[weitendorf]

No you're not making it up, this is exactly what some people are working on. Agent frameworks are starting to move towards "dynamic" service discovery/runtime introspection and composition-with-guardrails. Some keywords are "agent mesh", and the general marketing from AI companies about AI "inventors", and agent-driven interfaces like Google's a2ui (which is just a spec)

We recently started working on https://github.com/accretional/collector to serve as a dynamic proto ORM+CRUD server with search and discovery, and features for operating as a node in an "agent/service mesh". The idea is that you can create a uniform interface for data retrieval/search/APIs that lets agents dynamically register, invoke, or discover any data type or service, or write it themselves, then register it locally or share it.

It is feasible to do this stuff now actually, just a bit tricky because most LLMs aren't trained to operate this way without very explicit instructions for how to do so, and for collector specifically the API surface is probably too big. But I am pretty sure neither would take long to fix if enough people were adopting this kind of pattern.

[tokioyoyo]

That’s actually really cool, and makes sense in my head! This is somewhat how I imagined it, except my guess would be someone would fine tune a general purpose LLMs (somehow, as it is much cheaper than starting from scratch, idk?) to behave this way rather than instructing it all the way in. And whoever develops the framework would package it with the access to this fine-tuned LLM.

But yeah, what you guys are doing looks sweet! I need to get out of my ass and see what people are doing in this sphere as it sounds fun.

[weitendorf]

> fine tune a general purpose LLMs (somehow, as it is much cheaper than starting from scratch, idk?) to behave this way rather than instructing it all the way in

I'd love to do that too but there are basically three ways to teach LLMs how to use it afaik: with data created "in the wild" and a degree of curation or augmentation, or with full-on reinforcement learning/goal-oriented training, or some kind of hybrid based on eg conformance testing and validating LLM output at a less sophisticated level (eg if it tries to call an api that's not in the set that it just saw during discovery, the LLM is being dumb, train it out of doing that).

The thing is they are not really mutually exclusive, and LLM companies will do it anyway to make their models useful if enough people are using this or want to use it. This is what's happened already with eg MCP and skills and many programming languages. Anyway, if prompting works to get it to use it properly it validates that the model can be trained to follow that process too, the same way it knows how to work with React

[tokioyoyo]

I see, makes sense! I’ll try to keep up to see what you guys are doing and overcome the problems. Thanks a lot!

[fluidcruft]

My experience with LLM and agents has led to the opinion that a LLM-friendly codebase is actually a very human friendly code base.

[simonw]

Same here. So far everything I have found to help LLMs is just good practice generally: automated tests, documentation, clear issue descriptions, a neat commit history, well featured code etc.

[DANmode]

Documentation, aka any kind of architectural plan,

vs

“we’ll figure it out when we get there” human slop.