[simonw]

This one is pretty bad. This guy found a fake Facebook customer support phone number in a Google search, then asked the Meta AI chat in Facebook Messenger if the number he found was a real Facebook help line... and Meta AI said that it was. There's a screenshot of the chat in the article.

[idle_zealot]

The bad thing is that people still think LLMs can be trusted at all. Companies integrating them into their offerings are not helping the public adopt the correct mental framing of these tools as "plausible text generators".

[joe_the_user]

Companies integrating them into their offerings are not helping the public adopt the correct mental framing of these tools as "plausible text generators"

"Not helping" seems a wild understatement. "Deceiving people into taking the wrong frame" seems more accurate.

[Gigachad]

The general public is getting lied to constantly. HN users have a bit more context to see through the bullshit but the marketing getting pushed in people is that these AI tools are super genius incredible world changing tools that make everyone 100x more productive.

[dotnet00]

Even many HN users instantly resort to misdirection via comparisons to humans or nebulous upcoming AGI instead of acknowledging that we have to live with these limitations for the forseeable future.

[rootusrootus]

Maybe we have a bunch of users who primarily code in languages with duck typing. So that extends over to assessing the abilities of LLMs -- "talks like a human, therefore it is the same thing."

I'm only sorta kidding. I am surprised at the number of people who are comfortable with such a shallow conclusion.

[dorkwood]

This can be solved with more data. New tech like Windows Recall should be able to scrape enough of the world's data so that this sort of thing doesn't happen anymore.

[threeseed]

a) There is no evidence that it can be solved with more data.

b) Windows Recall data is never going to make its way into public models.

[dorkwood]

> b) Windows Recall data is never going to make its way into public models.

It doesn't need to. It just needs to make its way into OpenAI's models, which it will.

[RecycledEle]

> The bad thing is that people still think LLMs can be trusted at all.

LLMs are as trustworthy as humans.

Humans have been being wrong for about as long as we have been lying.

Whether you get information from a human or an LLM, check it.

I worry about the people who insist on credible sources rather than checking information for themselves. I think 80% or more of them are trolling me, but there are some who genuinely do not apply the Scientific Method to check facts in their everyday life. I truly feel sorry for them.

[MajimasEyepatch]

This is not true. Sure, humans can lie or get things wrong. But normal people will also admit when they don't know something. LLMs tend not to admit when they don't know something, and they use an authoritative voice that sounds like they know what they're talking about. To an untrained person, this can easily be misleading.

[dylan604]

> But normal people will also admit when they don't know something.

You'd like to think so, right? However, this isn't really a solid thesis. Decent people will admit when they don't know. Is that normal? I've worked with so many people that just do not fit that definition at all, to the point it just seems like that's the normal way to behave. Maybe I'm jaded grossly overweighting it, but it just seems I have been in way too many meets with too many arguments over something because someone refused to back down and admit their ignorance/arrogance wasted valuable time because of refusal to accept input from others.

[threeseed]

> However, this isn't really a solid thesis

Let's get 1000 random people in a surgery room and ask them to perform brain surgery.

You actually think that most of them will say "sure I know exactly how to do this".

Be serious.

[lazide]

Some people can admit when they’re wrong.

When was the last time Trump admitted he was wrong?

[MajimasEyepatch]

Nothing about Trump is normal.

[lxgr]

Even if that were true (I don't think it is): The more important distinction between humans and LLMs is accountability.

If a customer support agent gives you incorrect information, you can often hold the company liable for it (assuming you can prove it; I suppose there's a reason for why companies prefer certain support channels over others).

If an AI "lies" to you, you're largely on your own right now.

[Majromax]

Not necessarily. In Canada, a case in February (https://www.mccarthy.ca/en/insights/blogs/techlex/moffatt-v-...) held that Air Canada could be held responsible for incorrect information about a refund given to a customer by its chatbot.

Notwithstanding differences in jurisdiction, applying that idea to this case would rely on finding that Meta owed Gaudreau a duty of care that extended to the Meta AI chatbot.

It would be more difficult to make this claim if Gaudreau had asked the question of Google, since Google itself is not usually responsible for false information uncovered by its searches.

[lxgr]

That's going to be indeed an interesting question (also discussed in this sibling thread: https://news.ycombinator.com/item?id=40536860).

My gut feeling is that it should be possible for companies to distinguish an AI product (i.e. as something provided to customers like a search engine, as you say) from an AI "working for them", but I can see a lot more disclaimers showing up in Meta's various AI chat channels soon.

[markhahn]

did Meta present the AI as an official customer-service chatbot?

[lxgr]

What I see on WhatsApp:

"Messages are generated by Meta AI. Some may be inaccurate or inappropriate. Learn more."

Which leads to a pop-up further explaining that use cases include things like "creating something new like text or images".

I think it's going to be really interesting to see whether that's considered enough by courts, or if they'll take the position that these things pretend too well to be a real person to make such a disclaimer sufficient, similarly to how e.g. a brokerage can't disclaim "no investment advice" and then go on to say "but buy this stock, it's gonna moon tomorrow, trust me bro".

[layer8]

Look at the screenshot in the article. If a human Facebook representative would give that response, would you not trust them? And if not, how would you apply the Scientific Method to fact-check it?

[TuringTourist]

It would be nice to have a confidence level for pieces of information, like humans have

[Mawr]

In theory. In practice, every piece of information you can get from a human has mountains of context around it which lets you gauge the reliability of the information.

A skilled motorcycle rider explaining how to take corners in a widely watched youtube video, with hundreds of comments confirming the advice and several recommended videos from other riders that basically say the same thing is an extremely strong positive signal.

The same answer gotten from a magic AI answer box is just as likely to be right as wrong, 50/50.

[idle_zealot]

Good luck checking every fact you encounter with the scientific method (and making sure to repeat your experiments to ensure reliability, oh and don't forget peer review to evaluate your methodology). What is your proposed scientific experiment to test... what Facebook's support number is?

My point is just that credible sources are absolutely necessary for information to disperse. Nobody can afford to figure out the modern world from first principles.

[josephcsible]

It's not about intentional deception. LLMs are very confidently incorrect way more often than humans are.

[lxgr]

Eh, I've had the questionable pleasure of talking to first level support call centers a couple of times recently, and I wouldn't be so sure about that.

The number of times I've been told that yes, resetting my iPhone's network settings and reinstalling an app will resolve my billing issue or similar...

[CoastalCoder]

This reminds me of that recent issue with a Canadian airline, where (IIRC) a court ruled that their chatbot made a wrong, but binding, commitment to a customer.

I'm curious if a Canadian court would hold Meta liable for the man's losses in this case as well.

[dghlsakjg]

That was a very interesting case. The chatbot in question was not LLM based (the incident was pre-chatGPT in any case) and was simply parroting an out of date or incorrect policy that it had been explicitly programmed to do. It seemed to gain a lot more traction in the press because of LLMs. "Air Canada forced to honor terms and conditions on their website" is a whole lot less interesting.

This FB thing is a case of an LLM simply hallucinating without direct human intervention.

Very different cases from a computer science perspective. My hope is that legally, they don't get viewed differently.

If you outsource functions of your business to a third party contractor you are still responsible for what they do and say. I don't think we should allow companies to weasel out of their obligations because they were dumb enough to let a sentence generator loose in a way that it could make commitments.

[jessriedel]

Yea, it’s certainly a reasonable argument if the wrong information comes from the company itself.

[Animats]

That's an excellent point. That court decided that an AI agent was an agent in the legal sense. "Agent" is a legal concept - someone acting for someone else.[1] It's what allows employees to act for a company. Otherwise nobody could do anything without signoff from the top. There are limits to agency, but it's a rule of reason thing - you can assume a store clerk has the authority to sell you stuff, and someone whose job is to answer questions has the authority to answer questions. The company has responsibility for the agent's actions within the scope of their authority.

[1] https://en.wikipedia.org/wiki/Law_of_agency

[2] https://www.upcounsel.com/lectl-what-the-california-civil-co...

[lxgr]

The situation here is slightly different, though. Meta's AI in their various products is explicitly marketed as an LLM chatbot, not as a customer support channel.

Whether they've been diligent enough in making that distinction (and whether that's even possible) will very likely be determined in court at some point.

[hermitdev]

Yeah, headline is overly broad by just saying 'AI'. From just the headline itself, it'd be easy to write this off as "duh, this guy's a fool", but the AI in question here is from Meta, itself. And, not only is it from Meta, but it's the AI they've put in charge of support.

[jessriedel]

> And, not only is it from Meta, but it's the AI they've put in charge of support.

Does it say that in the article or somewhere else? I didn’t see that in the article.

[hermitdev]

You can see it in the screenshots.

[jessriedel]

It says “Meta AI”, but I don’t see an indicator that it’s labeled as providing support. On my device, it doesn’t say so, and is labeled as possibly “inaccurate or inappropriate “. (It still provides a bogus phone number.)

[throwaway48476]

I wonder if he has a legal claim like the Air Canada passenger who the AI quoted a ficticious reimbursement policy.

[astrange]

That incident happened before ChatGPT was released and probably didn't involve AI. Anyone can write a wrong customer support script if they try.

[p3rls]

We're going to see a lot more SEO scams coming from social media platforms now that Google is promoting places like reddit and quora. Even on rSEO you can see moderators there asking themselves questions from alt accounts subtly promoting themselves. It's dog shit scammers all the way down.

[echoangle]

I mean that’s kind of on meta, as a customer I shouldn’t really have to care about the internals of the company. If a disgruntled employee lies to customers, that shouldn’t be the customers problem either. To me, that’s all just a statement by the company.

[naveen99]

Meta ai is so bad. What did they really do with all those h100’s ?