[dontupvoteme]

I've only ever queried (very large) databases but my eyes always go a bit wide when i see statements that touch tables. They scare me. They scare me when i run them on an sqlite table i made 5 minutes ago for an experiment.

I see the problem as much, much more insidious and not the expected threat vector. The past few months many of us have seen these models become increasingly worse at keeping track of details and hallucinating.

They mix in information within their context window, and the cope that OpenAI has given us for their worse ability to generate good quality output is .... more context! Great.

So what happens when that context window (which you have no real idea how they're actually implementing it) has the concept of "DROP" in it? Or what happens when It's a long day, you looked over it and it's all correct, but in some buried inner query something changed? Probably it just costs some time to debug, bu..

Obviously there should be a few safeguards before that query gets executed but i never want to see an increasingly cheapening and more wide-spread black box like GPT be able to "speak" a word which in principle can cost 6-7 figure damages or worse.

We don't let actively hallucinating people brandish firearms for a reason