[kgeist]

How do you validate that the reports are correct? What if an executive makes a wrong business decision because the LLM wrote a wrong SQL query?

[pocksuppet]

https://thedailywtf.com/articles/The-Great-Excel-Spreadsheet

[BrentOzar]

> What if an executive makes a wrong business decision

I jokingly tell students, "We all know executives are gonna make bad decisions no matter what the data says. Might as well give them the random numbers more quickly."

[nananana9]

The same way we've always done it - glance at it and see if the numbers look like they're within an order of magnitude of what looks reasonable.

[fl4regun]

so what if there were some numbers in the report which are in actuality, an order of magnitude or two outside of what you think is reasonable, because something was wrong, but the AI agent reports something that looks normal?

[tremon]

So as long as the LLM only makes errors in the single-digit percentage range, everything is peachy. Make number go up, but not by too much.

[marcosdumay]

If you already know the report's numbers, why are you asking an LLM to generate it?

[nananana9]

Usually because you need something vaguely technical and authoritative sounding to push for a decision you're already made.