[jug]

Yes, AI in its current form is going to be a problem. I'm sure we haven't heard the worst yet. An AI may eventually kill a user.

I believe the heart of the problem is that corporations are riding a hype wave as long as they can, and an AI chat looks like super convincing, next level stuff thanks to the simple interface that hides the fact that you cannot communicate with this one as you would with a human being. You use natural language and it responds with natural language, which makes it not only convenient, but also dangerous.

There's money to gain on all this. While at the same time, hallucinations are an unsolved problem as well as making AI humble enough to realize and tell users that they just don't know. The combination of hallucinating, raising convincing arguments, being confidently incorrect, and not knowing the boundaries of your knowledge base, is a terrible one to let loose as officially sanctioned products.