[s_dev]

Why is this concern more important the what people interpret from the media, social media and the dissemination of information in general where lies and fabrications are also commonplace? Like surely people will always fall for nonsense, lies or fabrications and there is nothing that can be done about that.

[swatcoder]

As with all these discussions: accountability and consequence.

We can point at a media company, call out its vested interests, scream about its bias, protest in front of its office, sue it for slander and misrepresentation. We can call out individual personalities the same way. We can strive to drive the companies out of business and the personalities out of work, if we deem it necessary, and we can accumulate a paper trail that holds each one to account.

As neither individuals nor corporate entities, algorithms do not yet carry this kind of legal or public accountability even as we some start to hold them up as oracles. In most cases, failures of an algorithm are treated simply as bugs or user mistakes. Nobody is responsible for anything bad and the so the algorithm can persist and its vendor can shrug off their own responsibility by gesturing towards an perpetual development process instead of accepting consequence: "we work to make the algorithm better every day, try again tomorrow!"

[s_dev]

>We can point at a media company, call out its vested interests, scream about its bias, protest in front of its office, sue it for slander and misrepresentation.

Right but the previous election had Russian servers spinning up fake news websites that displayed straight up generated news. Again how do you hold them accountable? You can't the only defence against bullshit is independent thinking.

[Spivak]

Because LLMs strip away all the context surrounding the information it spits out that let you evaluate its trustworthiness. They're incredibly useful tools, I use them constantly when coding but I can do that because I know enough to validate the information and it happens that the cost of validating the output with the docs is shorter than reading them to find the relevant functions.

I wouldn't dare try to use an LLM for a chemistry question because I wouldn't be able to tell if it makes any sense or not. But if you're not a "tech person" and all you see is some company advertising their AIs as magical knowledge engines with disclaimer text that wouldn't pass accessibility tests, why wouldn't you assume they know their stuff? The Perplexity ads are bordering on negligent.

[evanmoran]

The difference is that web/social media is branded as an intelligent being you can ask any question of. We all agree the web is _also_ not reliable, but many people will think GPT / Gemini are verifiably accurate when they aren’t.

[BeetleB]

> We all agree the web is _also_ not reliable

You need to expand your circle a bit more :-)

[jrm4]

Not much different. Except that there's a dumb thing that humans are doing which is giving weight to the magic brainy computers.

[0xdeadbeefbabe]

All that training data would make even grep look smart. I'm just glad those magical forest creatures made the data in the first place.

[codr7]

I've found that people in general seem to trust computers more than humans, which made sort of sense for a while.

What they don't fully realize is that this is a completely different game; now the computer is just guessing, as opposed to following a deterministic algorithm to the answer.

And this misunderstanding carries the potential for pretty serious consequences, good luck getting that loan once a computer finds some arbitrary pattern and says no.

[6510]

If only it would tell you "You've criticized the war effort that day in 2004", in stead it will do parallel construction. The end game will be a kind of SEO for human profiles and we will live happily ever after by the best practice guide lines.