[adamddev1]

The lack of consent is real. As one of those viral commencement speakers who got booed said:

"Find a way to say yes."

My greater concern is that we are trading away hard earned truths for drastically inferred best guesses and slop that is just good enough to work. Search engine AI summaries flat out lie and deceive people all the time. LLM agents fudge over and cut corners. What people don't understand is the exponential damage that will be done as we keep baking these errors and untruths into everything we build, and build with.

[ragequittah]

>Search engine AI summaries flat out lie and deceive people all the time.

What was the alternative you were using that gave you 100% accurate information before? Search engines lied all the time as well. As did news sites and any other place you could go to for information. AI just makes it easier to get the information people were already consuming.

[atomicnumber3]

People always point this out like it's some kind of gotcha. "But things were wrong before, too!"

But it wasn't ON PURPOSE. The INTENT, by the people serving the search, was for the information to be correct. Algorithms tried to reward correctness, people would curate information, etc. Sure, bad actors tried to game it. But since the intent was for it to be correct, search providers fought back.

With AI, you're literally intending for there to be this chance - and it's very hard to gauge what percent it is because it depends tremendously on your query - that the result is just straight up fucking wrong. Google search results didn't used to have a "btw this might just be totally madeup" disclaimer on them, or even on the quick-answer box.

The intent matters so much.

(I think this honestly extends to code too, though I won't belabor that point in this text box.)

[jason_oster]

Between the elimination of the fairness doctrine, the rampant sensationalism (a phenomenon so common it has its own derogatory name: clickbait), the corporate ownership of media outlets, and the pervasiveness of advertising, I can't imagine how you got the idea that information sources were ever a public good. Let alone conflated the purpose of the intent. The intent has been to make money by any means necessary.

[happytoexplain]

This is fatalism via cherry-picking. It can't help anybody.

[mikepurvis]

Surely it's obvious, but authority matters. If I do a google search and then scan over a SERP and click through and read the answer I'm looking for, there are a hundred and one signals about where it's coming from. Is it a news page? The manufacturer's documentation? Someone's blog? A random forum post? If it's a forum post, are there are others participating, and if so what do they say about it?

All of this matters, and all of it is erased when it's just Google itself apparently telling me in black on white text above all the search results "here's the answer you're looking for ps. I'm an AI and can make mistakes, btw this space will sometimes secretly be an ad lol"

[adamddev1]

The AI summaries are of course another layer of lies. The lie about the sources (which may also be lies). Before the lie would have to be written by a human and published on some website. I'm not saying that was good. But it was more manageable than the exponential and automatic falsehood being baked into the fabric of the web now.

This is a similar logic to saying, so what if the LLMs introduce bugs, did humans not make mistakes before? Or, so what if I'm cheating, does no-one around me cheat? Corruption doesn't justify larger scale corruption.

[flerchin]

The difference being that stupid-as website was a signal users could filter. "I found that drinking gasoline is actually medicine on 4chan.org" is vastly different than "Gemini suggests drinking gasoline as medicine." One is the open internet being dodgy, the other is google with more resources than God.

[Ferret7446]

Is it though? You should treat everything you see on the Internet with equal skepticism, whether it is on ChatGPT or Reddit or even NYT.

[happytoexplain]

Why on earth should you treat everything anybody says (which is basically the equivalent of saying "everything on the entire internet") with equal skepticism? That's so unrealistic, how could that ever make sense?

[flerchin]

Yeah ok. How does gasoline taste?

[happytoexplain]

This is a super common fallacy about how people "should" feel about AI reliability ("people are wrong too!").

People are wrong/lie in different ways from AI. We have highly developed personal heuristics about where to place human writing on a gradient from 0% to 100%, based on the source, the topic, and a hundred other variables we don't even realize we're ingesting. Even without a way to verify, we are comfortable with this state of affairs. AI lies in more randomly distributed, unpredictable, confident ways. Even giving the benefit of the doubt that these falsehoods are rarer than human falsehoods, it creates a constant background of cognitive stress (FUD) and a feeling of indignation.

Further, the "who is wrong more often" question is complicated because we ingest human-created and AI-created data in different contexts. But it seems both evident and intuitive that AI is wrong more often, as long as you accept "I don't know" as not being wrong. You can ask it anything, and it will much more rarely say "I don't know" than a human who also doesn't know would. For example, if you accidentally ask it a question that contains a false implication, it will more often than a human just assume your implication represents reality.

Also, nobody claimed 100%. It's a red flag to write in black-and-white like that.

[customguy]

> What was the alternative you were using that gave you 100% accurate information before?

Why are these mega corps rushing to make money? Will they magically keep existing forever, or will they disappear into dust, along with everything and everyone else? So what would be lost if they were dissolved today?

> AI just makes it easier

Oh? So does it literally take zero effort, energy and time? Or is it actually not quite so easy? What, fundamentally, would change if instead of mumbling something into the mic to get half hallucination, half garbage back, users had to push a rock up a hill, with that rock tumbling down just before they reach the top?

[gonzalohm]

It's about the user of that information.

As a programmer, I'm comfortable judging the coding output of an LLM. But now, anyone can go and start building without any knowledge, and at first it may look fine, but you are creating software using a pretty weak foundation, bad maintainability, etc.

I think LLMs allow everyone to skip an important step of building anything, which is understanding how things work.

[clapthewind]

Why is that (framed as) a bad thing? We don't go around life understanding everything that we do because it would be exhausting.

Sometimes all you want to do is use something and it works.

[gonzalohm]

It depends on the application. A low stakes vibe coded personal project? Sure go ahead, that's a perfect use. A proof of concept? Also pretty good use.

For the main product of your company I think it's fair to say that people should know and understand what they are doing

[ragequittah]

You can understand deeply what you're doing and still make a huge amount of mistakes. This is what we're ironically (in terms of this conversation) seeing AI point out over and over now that companies have mythos in their hands. If I had to bet on it at some point we'll see the shift where if AI hasn't fully reviewed and edited code it's considered unsafe to use. Similar to how we all understand how once self-driving cars are good enough human driving should probably be minimized because we're so bad at it.

[Krssst]

The alternative before is that you knew where you were reading the information from. So you knew what level of trust to give each information.

AI summaries removes that ability from you, and even when it gives the source it may paraphrase it incorrectly just because LLMs are fundamentally unreliable. The level of trust to give LLM summaries is 0.

[dare944]

> AI just makes it easier to get the information people were already consuming.

This is clearly false. By now it's evident that AIs can hallucinate in ways that contradict their training data. That means that, beyond any errors or misinformation present in their input, AIs also naturally introduce their own level of error on top.

Beyond this, of course, AIs are much easier to manipulate during training, much more so than the Internet at large. So they represent yet another source of intentional deception aimed at the user.

[rightbyte]

Adding an lying abstraction just makes noticeing lies even harder. Google's summary even lies about the sources for the lies.

[adamddev1]

So true. The last 3-4 times I have checked the sources did NOT say what the Google AI summary said they did.

[mrguyorama]

The time's the AI summary are actually useful are when it repeats, verbatim, some website, usually wikipedia.

[jcgrillo]

Before, when working on my car I would google bolt torque specs and usually the first result was authoritative--generally a forum post from a reliable, professional mechanic quoting the factory shop manual. Now the first result is an AI summary, which is wrong roughly 25-50% of the time in my experience. Often it does incredibly stupid things like mix up ft•lbs and N•m. If I were to trust this, it could kill me (or worse, someone else).

Now I don't google at all anymore, instead I spend 10min flipping through the manual to find the spec. One might argue that I should have been doing this all along, but it sure was convenient to have basically a 100% reliable, fast alternative. Now that's gone.

[bigstrat2003]

Nobody has ever claimed that the available options before LLMs were 100% accurate. But they were more accurate than the LLM.