[andrewmutz]

I think it's a completely reasonable position that companies making self-driving cars and question/answer systems are legally liable for any errors.

But if you hold that position, you also have to be fine with companies not offering products and services in your country. AI systems will eventually be good enough (in 10-20 years) for companies to be able to deploy such systems with sufficient accuracy to afford the lawsuits. Until that time, such countries would just not have access to systems before they were bulletproof.

[NitpickLawyer]

> AI systems will eventually be good enough (in 10-20 years) for companies to be able to deploy such systems with sufficient accuracy to afford the lawsuits.

I doubt that will be the case, because of the long tail problem. (same with self driving cars and other ML related problems).

In fact, we have counter-examples today. Newspapers (even reputable ones) can't get it right every time, despite the fact that they have both trained people and in theory they're setup to catch that w/ reporters - fact checkers - editors. And still, from time to time, they get it wrong. (and I'm not talking about purposefully getting it wrong, just honest mistakes.)

What will likely happen with a ruling like this is that the answers will be hedged and legalesed and muddied up the wazoo.

[jojomodding]

Newspapers have mechanisms like corrections and apologies that can be used to "right" a published falsehoods.

This relies on me being able to find out if a newspaper lies about me, which is usually easy since we can all go buy the same newspaper. With AI it is much harder to find out that it has been telling potential customers wrong things about my business.

[naasking]

> Newspapers have mechanisms like corrections and apologies that can be used to "right" a published falsehoods.

Ineffective mechanisms. So if we accept ineffective mechanisms as sufficient redress in those spheres, why not here too?

[spwa4]

You don't realize how this works do you? One incredible power (executive) governments have is to let people get away with crimes. People don't realize they have this power, but they control the public prosecutor who can legally choose to do something ... or not.

You will find references to this in stories where the government lets people get away with murder, because of course, that's dramatic for a story.

But when "bigger interests" (ie. the Chancellor's bank accounts) are at play, just to name one example, China gets to distribute lead-painted children's toys in Germany and doesn't have to accept liability. Russia gets to import sanctioned natural gas over illegally constructed pipelines. Etc.

As to how this goes within the EU, in the worst (but common) case, is as follows: the government chooses a company, and refuse to sue them. They have a tendency to choose the worst possible company, like using Palantir for policing Germans [1]. Then you find out most of the Chancellor's grandchildren are working there.

[1] https://www.dw.com/en/german-police-expands-use-of-palantir-...

[ozgrakkurt]

> AI systems will eventually be good enough (in 10-20 years) for companies to be able to deploy such systems with sufficient accuracy to afford the lawsuits.

This doesn’t sound convincing. What AI and what company?

LLMs don’t seem like they will ever be reliable like that.

Self driving like waymo might be?

[MYEUHD]

> LLMs don’t seem like they will ever be reliable like that.

True. But you never know if / when there will be a new big breakthrough in AI, which will probably be based on a new architecture / paradigm, i.e. it won't be LLM-based

[jnovek]

LLMs will absolutely be like that. The speed this technology is moving at makes me certain, especially over a period of 10-20 years; 20 years ago I was bugging friends for a GMail invite and AI was a joke left to academics.

I think it will even be solved soon, like, within the next 18 to 36 months. Hallucinations are the biggest problem consumers have with LLMs and a solution to that would be instantly worth billions of dollars. I’m sure every company in this space is desperately trying to figure it out before everyone else.

A non-deterministic system will always make mistakes, but we’ll hit a target where LLMs make fewer mistakes than humans and that will be good enough for almost all applications.

[gilgamesh-OG]

I don't know if you can "fix" hallucinations without changing the fundamental architecture. The other factor in this article is that prior to the AI summary at the top, Google could simply state that it was an error on the part of the website owner. Now it is being held liable for whatever the summary states - even if it's more accurate, it can still be wrong enough times to be expensive.

[geon]

> I don't know if you can "fix" hallucinations without changing the fundamental architecture.

Exactly. "hallucinations" are not some special case. They are the LLM working as designed.

[jnovek]

They don’t need to be eliminated or “fixed”, they just need to be less frequent than human error.

[ozgrakkurt]

This is disregarding the entire mechanism by which LLMs work. How close to this ideal are the current frontier level ai is now? If you do a cost/improvement analysis does it look like it can reach a usable threshold?

I don’t know the numbers but as user, it seems impossible for it to be useful without expert review. It is also debatable if it brings any value when you consider the cost of building and using LLMs and the time of expert. Also need to include the opportunity cost the expert is spending on reviewing slop instead of creating work themselves and the long term consequences of this on the expert himself

[jnovek]

Hallucination rates go down by a few % with each new model generation, although some milestones have seen regression e.g. reasoning models are just overall worse. It is a bit hard to measure and compare across generations because the tests have to change in order to stay ahead of training data but they are generally improving. Check out HaluHard, it’s a stress test hallucination benchmark.

https://halluhard.com/

There will never be a time where there are zero hallucinations because of the non-determinist nature of LLMs, but eventually the frequency of hallucinations will be so low that it doesn’t matter. If the robot makes one mistake for every ten a human makes, that’s coming out ahead (depending on the nature of the mistake, of course).

Also I’m not making any value judgement about the technology and how it’s used and, frankly, I don’t really appreciate the assumption that I am. I’m fucking job hunting right now and it’s a hellscape thanks to LLMs.

I’m just being real about where the tech appears to be going based on its current trajectory and my experience in the industry. There seems to be a lot of cope going around that these things won’t ever be good enough to take our jobs. They will, and sooner than any of us are ready for. Leadership is fine with slop as long as it ships, the tech as it stands today doesn’t have to be much better to reach that standard.

[NicuCalcea]

Who would argue for offering self-driving cars before they're ready and safe? As a cyclist and pedestrian, of course I don't want them in my country if nobody's going to be liable when they run me over. Let them work out the kinks on Americans since they're so eager to be on the cutting edge of progress.

[Swizec]

> Until that time, such countries would just not have access to systems before they were bulletproof.

Correct, most jurisdictions do not allow businesses which cannot be held liable for their actions. This is pretty core to a modern society.

Imagine if a company selling Knicks tickets was not expected to then actually provide said tickets and there was simply nothing you could do about it. Oopsies our sales page is for entertainment purposes only

To be fair, the internet has spent some 30 years figuring out how this works and it’s still not fully resolved. For the most part we’ve agreed that companies must follow the laws of both where they live and where they operate. This wasn’t always obvious!

[scottyah]

Almost all jurisdictions allow businesses which cannot be held liable for all their actions. Imagine losing you house because someone decided to smother themselves in your infused cooking oil and light themselves on fire.

Businesses are made to make it easier to share profits and responsibilities when trying to fulfill users wants. Laws are made to offer protections for consumers (because nobody has time for common sense), but at the end of the day the consumer has to take responsibility or no products can be made. If you're too fat for a chair, it's on you to find or make one that works- not every product is for every person. Laws only stop chairs being sold that are too dangerous for anyone.

[autoexec]

> Almost all jurisdictions allow businesses which cannot be held liable for all their actions. Imagine losing you house because someone decided to smother themselves in your infused cooking oil and light themselves on fire.

How would that be the company's action? The only way the business might be liable is if they advertised their product as safe to use when lighting yourself on fire or if there was already some law that required them to warn customers not to light themselves on fire while using the product and they ignored that law.

In this case, it's not about what somebody else did. It's what Google did. There were already laws against lying about companies by saying they did illegal things when they didn't, google broke the law, so that's what google got in trouble for.

Consumer protection laws aren't there to replace common sense, they're there to prevent things like outright fraud and poisonings/murder.

[kiviuq]

I see an economic and social problem:

Freedom as in freedom of private property can only be guaranteed by the State. The State watches over its own population and makes sure that private property, i.e. capital and work, is made productive, so people go to work, businesses make profits, and everybody pays taxes. Taxes are the State's prime source of income.

When all these million of private interests collide, which they are bound to do, the State provides a jurisdictional system that has to decide between those private interests and the State's own interests.

E.g.: If a business owner refuses access to medical patents or to lower prices and safe potentially people's lives, the State has to decide between that immediate interest and its own interests, which is protecting private capital, as its source of income. Since I'm in Germany: In the emission scandal Volkswagen didn't just physically harm people, VW violated the private property of millions of customers. Despite that, the German State sided with Volkswagen the larger capital and did nothing. During Corona, the German State refused to open patents for a limited time to help safe people's lives in poor countries. Doing so would've violated the interest of private capital, so it refused. In contrast, if I as an individual refuse to help somebody in an emergency, the State would either fine me or put me in jail. In this case, people's lives become the State's prime interests, because they are also the State's source of income, as a productive workforce.

[chias]

Sounds like a win win to me

[gblargg]

To remove the choice from responsible people who can understand that LLM answers are not to be trusted with anything important?

[sham1]

If our standard for laws would be that "well no reasonable person would do this/believe this" then nothing would be illegal, there'd be no need to label any product as potentiality harmful, etc.

Do you really want to go there? That everything in the world would have a literal "caveat emptor" attached to it?

[gblargg]

I thought Google labeled its AI summary with a disclaimer already. I don't want companies to be forced to only offer safe-for-children services.

[bergen]

And the european consumer doesn't want harmful products to be beta tested on the public.

[9dev]

It's unbelievable how lightly some people hand over the tools for mass manipulation to a single corporation in the name of freedom of all things. We're not talking about a laser pointer here.

[soco]

Heck even laser pointers are regulated, now that we're thinking about it.

[sham1]

There is a disclaimer, yes, but you have to admit that it's pretty shit, innit? I mean for one, it's about the size of a human hair, and at least when I tried it, the disclaimer came up only when I clicked the "Show More" button. It might admittedly show up earlier if the response is shorter, admittedly I don't know. Also personally I'm a bit uneasy with the idea that just with a simple disclaimer they could avoid any and all liability. Not your argument, I know, but still.

As for not wanting to force companies to release only "safe-for-children products", I do actually agree. However I consider it to be a matter of degree, and in this case for example, I think that if nothing else, Google should say the very least make the disclaimer a bit more prominent and maybe tweak the model so that it's not quite as confident in its claims in the AI Overview.

[ben_w]

> As for not wanting to force companies to release only "safe-for-children products", I do actually agree

That would be nice, but as every effort to restrict kids from using software which are not safe-for-children keeps getting condemned for being invasive surveilence, and every effort to stop kids getting the hardware instead gets condemned because of how much of society is now built on assumption everyone has a phone…

Something has to give.

Dunno what, but something.

[gblargg]

Yeah, they could make it more prominent at the top. I would be fine if it said that "AI may give totally wrong answers" but that would never happen.

[wsng]

The harm was not done to the readers of the AI generated response, but to the defamed companies.

And yes, it is ok to remove choice if the existence of that choice violates other person’s rights.

Google can continue offering that choice if they make sure nobody is defamed.

[bergen]

So we deploy a technology no one should trust to the general public, for what exact reason?

[autoexec]

So a very small number of people can get very rich off of the suffering of a massive number of other people I guess

[buellerbueller]

Venture Capital 101! Welcome to Y Combinator :)

[mrspuratic]

Land grab strategy 101.

[polotics]

Do you mean the responsible people who will ensure their algorithms can be trusted with the important task of acting in the best interest of said people? Try and get a defamatory statement about google from the AI search box.

[ndsipa_pomu]

There's plenty of examples of supposedly responsible people using LLM answers verbatim without any kind of overview by themselves e.g. https://www.theguardian.com/technology/2025/jun/06/high-cour...

[phatfish]

I have to fight with my family members when they "Google" something, read the top AI slop result, and I ask which page it came from. They believe what is on the Google landing page, and actually I don't think that is a naive assumption. Google has pushed itself as the information oracle, now they are delivering slop as the first result. It's a bait and switch.

[circuit10]

I think they should just have to properly explain how AI tends to make things up when it doesn’t know, and that it’s good for coming up with ideas or suggesting directions for research but that you shouldn’t rely on it, because currently their advertising makes you think you can rely on it

The “AI can make mistakes” kind of disclaimers they hide in the corner don’t really cut it

[input_sh]

It's not even in the corner, it's completely hidden by default. You have to click on "show more" to expand the AI barf to see it.

[autoexec]

> But if you hold that position, you also have to be fine with companies not offering products and services in your country.

What sounds like a win to me. I certainly hope my country makes it dangerous for companies to break the law and/or harm the public with shitty products that aren't ready to be released legally/safely.

[hypfer]

> But if you hold that position, you also have to be fine with companies not offering products and services in your country.

Well.. I mean.. yeah? I don't think this is as bad as you think it is.

Have you looked at SV and its product offerings recently? It's mostly just enshittified gamified value extraction that doesn't respect the user at all.

"If you do not let us do all this the way we want, we will take away your ability to use our shit" hits different when the "shit" in that sentence is actually just "shit".

[ben_w]

I'm half-remembering a now-old satire along the same lines has Germans wondering why having Google Street View work in their area also requires internal photos of their apartments.

[TonyTrapp]

This one? :) https://www.youtube.com/watch?v=OMFBuHsKXb0

[ben_w]

Genau :)

[lelanthran]

> I think it's a completely reasonable position that companies making self-driving cars and question/answer systems are legally liable for any errors.

Well, for cars anyway, the manufacturer was always liable for the car doing something wrong (example: driver changes the volume on the radio, and that disables the brakes).

It's just that techbros want an exception to this rule if the car is self-driving.

I see no reason for an exception to this rule.

[kiicia]

why offer expensive service when it's effectively useless and only add to cost and amount of work? what else they offer, "summarize my text" and "generate custom emoji"? I can live without that...

for now only volvo accepts liability, and only for "slow crawl mode"

[intended]

Uh, yeah of course ?

Let someone else sacrifice the safety of their populace.

Heck - self driving is the fastest way to authoritarian government in practice. I’m surprised more people on HN haven’t cottoned on to that fact.

A self driving system will naturally build networks to share road state.

This network will eventually shift over to the government having the ability to manage how traffic should move during emergencies.

And at that point the government can easily decide where your car should go.

The inevitability of this outcome is blindingly obvious.

It’s highly beneficial to let other nations experiment and simply be followers.

[Forgeties79]

Plenty of products are legal in some countries and not in others.

[buellerbueller]

Perfectly acceptable to me.

[bloppe]

Most of the time, human beings driving cars with their own hands and eyeballs are not "liable" for their errors (unless they can be proven negligent or drunk), unless you count their insurance going up. Most car accidents do not end with anybody getting arrested, or sued, or anything like that. Insurance pays out, premiums go up, case closed.

If Waymo can be proven negligent or something, then sure, bleed em dry. But as long as they're acting in good faith and significantly reducing overall road fatalities per mile driven, I think it's actually pretty unreasonable to try to hold them to such a high standard you end up subjecting society to more of the higher fatality rates caused by humans.

[jameshart]

An at-fault driver’s insurance pays out because they are liable. Your insurance covers your liability. That’s why you need insurance.