[leptons]

It's fine if LLMs are used casually, for things that don't affect anyone but the user. But when someone plugs an LLM into Social Security or other governmental bodies to take action on real human beings, then disaster awaits. Nobody is going to care if the LLM got it wrong if you're just chatting with it or writing some wonky code that doesn't matter in the real world, but when your government check is reduced or deleted by an LLM that is hallucinating, then the real problems start. These things should not be trusted with anything but the least consequential actions an individual would use it for.

[gte525u]

^This - we're trying to use one to partially automate some system engineering type activities.

It's great for reviews where any given reviewer could be expected to have a misunderstanding of certain details or skip a section (RAG somewhat helps this) - but it's frustrating for artifact generation where missing details cascade through the project.

As great as the technology (right now) it seems so far from reliable business process automation.