[isoprophlex]

An anecote that helps you maybe:

I do contracting work, we're building a text-to-sql automated business analyst. It's quite well-rounded: it tries to recover from errors, allows automatic creation of appropriate visualisations, has a generic "faq" component to help the user understand how to use the tool. The tool is available to some 10.000 b2b users.

It's just a bunch of prompts conditionally slapped together in a call graph.

The client needed AGENTIC AI, without specifying exactly what this meant. I spent two weeks pushing back on it, stating that if you replace the hardcoded call graph with something that has """free will""", accuracy and interpretability goes down whilst runtimes go up... but no, we must have agents.

So I did nothing, and called the current setup "constrained agentic ai". The result: High fives all around, everyone is happy

Make of that what you will... ai agents are at least 90% hype.

[bradarner]

The hype of Agentic AI is to LLMs what an MBA is to business. Overcomplicating something with language that is pretty common sense.

I've implement countless LLM based "agentic" workflows over the past year. They are simple. It is a series of prompts that maintain state with a targeted output.

The common association with "a floating R2D2" is not helpful.

They are not magic.

The core elements I'm seeing so far are: the prompt(s), a capacity for passing in context, a structure for defining how to move through the prompts, integrating the context into prompts, bridging the non-deterministic -> deterministic divide and callbacks or what-to-do-next

The closest analogy that I find helpful is lambda functions.

What makes them "feel" more complicated is the non-deterministic bits. But, in the end, it is text going in and text coming out.

[BrandiATMuhkuh]

Do you have some advice on how to build the structure on how to move from one prompt to the next?

Are you using a separate state manager + function calling so the LLM knows where it is?

[isoprophlex]

You can model it as a state machine, where the LLM decides to what state it wants to advance. In terms of developer ergonomics, strongly typed outputs help. You can for example force a function call at each step, where one of the call arguments is an enum specifying the state to advance to.

Shoot me an email if you want to discuss specifics!

[beernet]

Am I the only one who finds these types of comments arrogant? I mean, we get it, you know better and have been doing this for a long time and so forth...Sometimes I feel like it's just about relativizing whatever tech is popular right now. Just to come back two years later and say "oh well I've been telling people about this cool tech two years ago!"

[JTyQZSnP3cQGa8B]

Give a counter example then. I’ve been doing this for years: people want the hot new thing even if it’s the worst idea, you rebrand it, and everyone is happy. Then a few months later, people praise you for not having implemented that bad idea.

[th0ma5]

100% agree. I'm not sure what they're trying to convey even.

[SebaSeba]

Sounds awesome. :D For real, the anecdote is hilarious and I find it easy to believe but also sounds cool what you are working on.

[isoprophlex]

Well you work in the field for a while, and you accumulate anecdotes of colleagues dropping tactical sleep(5000)'s so they can shave some milliseconds of latency each week and keep the boss happy.

I love those stories but I could never do that with a straight face. However, the AI field is such an uphill battle against all the crap that LinkedIn influencers are pushing into the minds of the C-suite... I feel it's okay to get a bit creative to get a win-win here ;)

[simonw]

Love that. Reminds me of a time I was asked to build a "machine learning algorithm" driven recommendation system... and eventually I realized that delivering a recommendation system based on one big BM25 search query was fine, and the people asking for it to use "machine learning" didn't actually understand or care about the difference.

[isoprophlex]

Haha yes, the LLM era is "data science is the hottest new job" all over again.

I guess everything with an algorithm in it is AI if you look at it from enough of a distance...

[DaiPlusPlus]

I like to call it “Artificial AI”.

[arresin]

It's nice to combine the two but the ranking takes tuning.

[philipodonnell]

I’ve been doing a lot of work on semantic data architecture that better supports LLM analytics, did you use any framework or methodology to decide how exactly to present the data/metadata to the LLM context to allow it to make decisions?

[isoprophlex]

A pre-processing phase does a lot of heavy lifting, where we stuff the table and column comments, additional metadata, and some hand-tuned heuristics into a graph-like structure. Basically using LLMs itself to preprocess the schema metadata.

Everything is very boring tech-wise, using vanilla postgres/pgvector and a few hundred lines of python. Every RAG-searchable text field (mostly column descriptions and a list of LLM-generated example queries) is linked to nodes holding metadata, at most 2 hops out. The tool is available to 10.000 users, but load is only a few queries per minute at peak... so performance wise it's fine.

[philipodonnell]

Enhancing the comments on the existing data model seems to be the most common approach for sure. I'm implementing this as a data architecture at several clients and I've found creating a whole new logical structure designed for the LLM is really effective. Not being bound by the original data model lets you solve several problems related to the "n-hops" question, avoiding needing the comments, and the semantics of how data engineers define columns. Some more details here [1], but obviously you can implement this totally yourself by hand.

[1] (https://github.com/eloquentanalytics/pyeloquent/blob/main/RE...)

[moltar]

Is the tool public? We are looking for a solid text to sql tool that works with Athena.

[isoprophlex]

Sadly, no, it's a walled-off customer facing tool integrated into one of my client's B2B business intelligence portals.

Hope you can find a tool; the big data players are of course jumping on this (snowflake, databricks, they all talk about their text-to-sql tools).

If you have the budget and want something bespoke built that has some magic sauce tuned to your exact problem field, send me an email!