[ziml77]

What's the danger? It can see the schemas to help it generate the queries but it can't run anything on its own. Also you have to give the application credentials to an AI provider for the feature to work. So, you can just not do that.

[adamas]

There is no need of potential dangers to not want to have non-deterministic features in an application.

[justinclift]

> What's the danger?

Hallucinated ideas about what needs doing, what commands to run, etc.

So, data that's no longer reliable (ie could be subtly changed), or even outright data loss.

[Natfan]

just don't accept bogus changes it suggests? this is why having a human in the loop is a very good idea

[justinclift]

Yep, I fully agree. It'd be awesome if all (or even most?) of the users were immune to automation bias though. :)

https://en.wikipedia.org/wiki/Automation_bias

And hopefully the users who are using this aren't YOLO-ing it. Though I'm pretty sure we're all expecting there will be a sizeable percentage who are.