[ekianjo]

> If LLM is a black box by definition and there's no way to make it consistently work correctly, what is it good for?..

many things are unpredictable on the real world. Most of the machines we make are built upon layers of redundancies to make imperfect systems stable and predictable. this is no different.

[habinero]

It is different. Most systems aren't designed to be a slot machine.

[ekianjo]

Yet RAG systems can perform quite well, so it's a definite proof that you can build something reliable most of the time out of something not reliable in the first place.

[vrighter]

only if you lower your standards as to what "quite well". The biggest con by the AI industry so far is convincing people that 90% is somehow "quite well".

90% is only enough for uninformed people to buy into it, and fuel the hype train. 90% is low enough to be pretty much unusable in most production environments.

This is like coding FizzBuzz but only emitting Fizz, Buzz or a number, and skiping FizzBuzz, then claiming that your system is 93.3% accurate because only every 15th output is wrong. 93.3% is utter crap.