[sswatson]

LLMs are pretty bland but they don’t just summarize the top Google result. They can generate correct SQL queries to answer complex questions about novel datasets. Summarizing a search engine result does not get you anywhere close to that.

It may be fair to characterize what they’re doing as interpolative retrieval, but there’s no reason to deny that the “interpolative” part pulls a lot of weight.

P.S. Yes, reliability is a major problem for many potential LLM applications, but that is immaterial to the question of whether they're doing something qualitatively different from point lookups followed by summarization.

[otabdeveloper4]

> They can generate correct SQL queries to answer complex questions about novel datasets.

"Correct" is a big overstatement, unless by "SQL" you mean something extremely basic and ubiquitous.

[coderaptor]

The output can be explicitly constrained to a formal syntax (see outlines.dev).

For many cases this is more than enough to solve some hard problems well enough.

[cruffle_duffle]

Honestly I think the reason it is “extremely basic” is because while it has been trained on “the entire internet” it doesn’t know anything about your specific database schema beyond what you provided in your prompts.

If these LLMs were cheap and easy to train (or is it fine tune?) using your own schema and code base on top of its existing “whole internet” training data… it could almost certainly do more than just provide “basic stuff”.

Of course I think the training for your own personal stuff would need to be “different” somehow so it knows that while most of its training is generalistic the stuff you feed it is special and it needs to apply the generalist training as a means for understanding your personal stuff.

Or something like that. Whatever the case is it would need to be cheap, quick and easy to pick up a generalist LLM and supplement it with the entirety of your own personal corpus.