[nonethewiser]

>The trouble is that it works just enough to be dangerous.

This is a major theme with LLMs. When they first came out you'd see it randomly returning garbage in the middle of an otherwise good output maybe 30% of the time. You knew you had to go through it with a fine tooth comb.

Now it's more like 3%. And you just gloss over it.

I think you are right about Text-to-SQL being a trap. In this case the deficiencies are unaccaptable.

But elsewhere? I think we are going to see the "customer service effect" applied all over the place. I am referring to the downward trend of customer service. Where quality was eschewed for scale. We went from highly competent agents providing individual feedback to hard to reach agents with no agency. It scales, but the bar has been lowered significantly. You get less customer service.

I think AI begs for the same tradeoff. Settle for less because it makes some things easier. Which of course makes other things much more complex or even undermines the entire premise.

[zurfer]

Reliability is the important dimension to focus on, but what is your baseline? I've worked in data and me and my colleagues regularly had to confront bugs (on many levels) that were communicated to end users