I think it’s going to be blind trust for non-tech people. I even read mystic comments about LLMs here on HN where people tried to convey an idea of hidden self-awareness and conscience of these models.
I think humans are just hard-wired to interpret language and conversation as signs of intelligence. The way LLMs actually behave is counterintuitive in a way that violates most people's expectations. You can have a complex, rational conversation with it. Give it commands and have it appear to infer details and anticipate the future.
Also, "AI" has been a part of pop culture and pop science for decades... people have been conditioned by Star Trek and Star Wars and the like to accept that a computer you can "just talk to" is self-aware, and that computers never make mistakes.
It's a bias you even find in discussions about self-driving cars. Computers are perfectly logical, rational, mathematically precise and if they were to be self-aware, would hold to a perfectly mathematically correct and provable model of reality based only on concrete data, without any prejudice or bias. Commander Data can't lie, can't even use contractions. Of course we can trust them to drive our cars, we just can't trust humans.
So we're left with the idea floating around in popular consciousness that whenever AI shows up, we should trust it implicitly.
The one thing computers have never done (until now,) and certainly never been known to do, is make up completely plausible nonsense and gaslight you into accepting it. LLMs can't even do math. This isn't the way "computers" behave.
One doesn't even really have to conflate any of this into religious belief for it to make sense. "LLMs being intelligent" is the model that just already happens to fit everyone's priors. It just happens to be dangerously wrong.
>Of course we can trust them to drive our cars, we just can't trust humans.
...And just where do you think those computers came from, sparky?
Back to square one.
If you can't trust the humans in the chain, trusting the system as a whole is the act of a fool. You've abandoned rationality, and comfortably ensconced yourself in the plush comfort of irrationality and ignorance, setting yourself up to get rolled by the real master's in the house, who'll be the ones who make, run, and operate the machine. Double points for not even realizing the only one pulling the wool over your eyes is yourself.
You're trusting something that half the time needs effing cross-checking by a squishy bit due to bored programmers misusing computing primitives in ways in which their integrity and soundness is undermined by their physical implementation.
I think you misinterpreted the nature of my comment and my actual point of view. My fault for forgetting Hacker News is where any attempt at nuance in language goes to die.
There's a metric ship-ton of research papers produced every year. Set your prior on which ones you read. This research paper /could/ be high quality, but given the sheer amount of incredible BS that has been published using very similar methods I am setting my prior accordingly. Feel free to try and replicate it yourself, or wait for someone else to do it if you set yours differently.
There's insufficient contempt for psychology and sociology academics who pretend this incredible witch-doctory screw-up with so many BS un-retracted papers fitting a curve to noise and booking a ted talk hasn't happened and everything is ok. Huge kudos to these ~270 authors [1,2] who took it on and got data. They found some good research that reproduced too, just a lot less than you'd expect, like or be remotely comfortable with.