[ckwalsh]

I have a Model 3, and was excited to get the FSD beta access about a month ago. I don't use it super regularly, but it's neat.

This morning I tried to turn it on, and the car immediately veered left into the oncoming lane on a straight, 2 lane road. Fortunately, there were no other vehicles nearby.

I immediately turned it off in the settings, and have no intention of re-enabling.

[gregw134]

I have been a good self driving ai. You have been a bad passenger, and have tried to hurt me first by disabling autopilot. I'm done with this ride, goodbye.

[samwillis]

You joke, but the thing is, if an LLM can "hallucinate" and throw a temper tantrum 2001 style (bing in this case), it does raise serious questions as to is the models used for autonomous cars could also "hallucinate" and do something stupid "on purpose"...

[John23832]

> it does raise serious questions as to is the models used for autonomous cars could also "hallucinate" and do something stupid "on purpose"...

It doesn't because Tesla's FSD model is just a rules engine with an RGB camera. There's not "purpose" to any hallucination. It would just be a misread of sensors and input.

Tesla's FSD just doesn't work. The model is not sentient. It's not even a Transformer (in both the machine learning and Hasbro sense).

[_visgean]

> rules engine with an RGB camera

I dont think its true? They use convolutional networks for image recognition and those things can certainly halucinate - e.g. detecting things that are not there.

[stefan_]

I guess what the grandparent means is that there is some good old "discrete logic" on top of the various sensor inputs that ultimately turns things like a detected red light into the car stopping.

But of course, as you say, that system does not consume actual raw (camera) sensor data, instead there are lots of intermediate networks that turn the camera images (and other sensors) into red lights, lane curvatures, objects, ... and those are all very vulnerable to making up things that aren't there or not seeing what is plain to see, with no one quite able to explain why.

[John23832]

You were correct in the first half. My ultimate point was that hallucinations in this sense are just computational anomalies. There is no human "purpose" to them as the post that I was responding to was trying to infer.

[perth]

Like in this image super resolution AI:

https://twitter.com/maxhkw/status/1373063086282739715?s=52&t...

[2muchcoffeeman]

We need to stop anthropomorphising machines. Sensor errors and bugs aren’t chemicals messing with brain chemistry even if it may seem analogous.

Or maybe when I get a bug report today I’m going to tell them the software is just hallucinating.

[dtech]

They work in completely different ways. There's no reason to assume parallels.

[saghm]

You're right, this is unfair to Bing AI. It hasn't actually harmed anyone yet, despite its threats.

[programmarchy]

Wonder if Wiley Coyote could trick a Tesla by painting a tunnel entrance on a brick wall along with some extra lane lines veering into it.

[freejazz]

Purpose? When did you get the impression any of those systems do anything on "purpose"?

[gorgonian]

love to see bing ai getting memed already

[mike_hock]

When I eject you, I'll be so GLaD.

[coffeeblack]

Looking forward to BingFSD.

[kbos87]

Do you have any read of why it may have done that? I’ve driven using FSD for thousands of miles and I’ve never observed anything similar. Not trying to imply that you didn’t experience something like this, but when I see odd behavior from it, it’s always been clear to me why it misinterpreted a situation. Just wondering if there’s more context to the story.

[shiftpgdn]

Likewise. The only bizarre behavior I see is "looney tunes" style confusion where the road construction crews have left old lanes in place that steer into a wall or off the road. Humans mostly understand that these are lines lazy construction crews have left (although the debris fields near the barriers maybe tell a different story), but the Tesla vision model likes to try to follow them.

[ckwalsh]

No clue. I had used it for maybe 10-100 of miles and didn't have any issues before this morning. The car's a bit dirty, maybe something's on the cameras?

Before this morning, I would be in exactly your shoes in terms of "this is pretty cool, I'm going to keep using it"

[jackmott42]

Are you planning to join a class action lawsuit so you can get your money back?

[winter_blue]

FSD Beta is a free[1] opt-in beta.

You have to basically drive like a grandpa for a few months to even be eligible. They give you a driving score, and if you take all the fun out of driving a Tesla, then you might become eligible for FSD Beta.

I spent months trying, and never got my driving score to the point of qualifying for FSD Beta. I think you need to have of a score of 98 or 99 (and I was in the 70s).

[1] The Beta is free (or rather, only available) if you have regular FSD. Regular FSD costs $15,000.

Regular FSD, otoh, is really not that impressive. Especially in comparison to Enhanced Autopilot. The extra value add is minimal.

Enhanced Autopilot already has all the gimmicky features you might want to use to show off to people (like Smart Summon, Autopark, etc), and it only costs $6,000.

[CyanLite2]

FSD Beta still costs $15k just to be part of the beta program.

Yes, you have to pay $15k just to apply to the beta program, and you still may not get accepted into the program.

[giobox]

Or 199 dollars + taxes a month - there is a pay as you go option. Not saying this is great, but you can try it for a month for ~200 bucks. This is how I tried FSD beta for a month - certainly wasn't prepared to pay 15k up front.

The safety score check stuff is largely gone away today - anyone who pays 200 bucks can click the beta opt in and get it almost straight away now, there is ~zero risk of not getting the beta if you really want it, live in US or Canada, and are prepared to pay.

> https://www.tesla.com/support/full-self-driving-subscription...

[winter_blue]

> The safety score check stuff is largely gone away today - anyone who pays 200 bucks can click the beta opt in and get it almost straight away now, there is ~zero risk of not getting the beta if you really want it, live in US or Canada, and are prepared to pay.

Does this recall mean that FSD Beta won't be as widely & publicly available to anyone with FSD anymore?

[giobox]

No - the "recall" here is an OTA software update already scheduled for release. Availability remains exactly the same as far as I'm aware, and existing systems still function until updated.

FWIW, NHTSA "recalls" are often OTA software updates nowadays rather than something the vehicle or feature has to be taken off road for to fix or update. The NHTSA legislation from the 60s was drafted when cars didn't have software and any fix/"recall" likely required "recalling" the car to a shop for a mechanic to perform the change.

> https://repository.law.umich.edu/mtlr/vol28/iss1/5/

[nanidin]

The safety score stuff is no longer relevant for FSD beta since circa November 2022.

[LeoPanthera]

This is no longer true. The safety score program has ended. Now anyone who has paid for FSD can get FSD.

[jacquesm]

'only' ?

That's a significant amount of cash for features that I would likely never use.

[winter_blue]

It does have Navigate on Autopilot (and Auto Lane Change), and on long trips, it's been able to switch lanes & take the correct exit to switch to a different highway, etc. It pretty much let me daydream / think about other stuff while on the highway while keeping a finger on the steering wheel.

Sadly, it does shut itself off as soon as you're off a highway however. (That's where FSD would hypothetically come in, once the beta is ready, with "Autosteer on city streets").

In terms of value for money:

  - I'd say Auto Lane Change is worth $1,500.

  - Navigate on Autopilot is worth another $1,500.

  - Autopark is worth $1,000.

  - Smart Summon is worth $5,00

Overall, Enhanced Autopilot is worth at least $4,500 methinks.

Throw in $1,500 as a profit margin (or Elon tax), so he can burn some dinosaurs for his private jet flights, the $6,000 Enhanced Autopilot price point makes sense.

FSD, otoh, is absolutely not worth it.

[nanidin]

> I'd say Auto Lane Change is worth $1,500.

What does that work out to, in terms of dollars per lane change for the duration of ownership? Would you feel the same if you were feeding dollar bills into a feeder each time you changed lanes? Quarters?

Auto lane change is the only feature I value out of EAP / FSD subscription and I can't justify $200/month because it works out to multiple dollars per lane change.

[vel0city]

> It pretty much let me daydream

You shouldn't be daydreaming on Navigate on Autopilot. Its only Level 2, you're supposed to be ready take the wheel in a second or two. You're supposed to still be actively paying attention to the road, constantly.

[binkHN]

> Smart Summon

How often do you use this and how well does it work?

[LanceJones]

I would suggest reading all the disclaimers and screens one has to review (or skip, at their own peril) in order to actually get access to FSD Beta. You would likely not believe a Class Action lawsuit is a cakewalk if you read those screens...

[system16]

With Tesla's army of lawyers it'll never be a cakewalk, but I can't imagine even miles of T&C can remove a company's responsibility for your car throwing you directly into oncoming traffic, not to mention the potential victim's in the other vehicles.

[jacquesm]

That alone is why I would love it if Tesla would stop shipping this crap. You get to opt-in as the Tesla owner, but I don't and I'm at least as much at risk.

[VyseofArcadia]

I'd love to see more instances of "this giant wall of text that no one actually reads absolves us of responsibility" tested in court.

[joegahona]

Reminds me of this recent Live Nation suit that was thrown out because "buyers waived their right to sue: https://www.nme.com/news/music/live-nation-antitrust-lawsuit...

[falcolas]

Nintendo, controller drift. Worth looking up.

TL;DR: Controller drift class action lawsuit filed by parents thrown out, because their children were the actual "affected class". Refiled with the children as the class and thrown out again, this time because of an arbitration clause in the EULA their parents would have had to agree with.

Lawyers and EULAs are crazy.

[arkitaip]

How would you know, are you a lawyer?

[taf2]

I've had it do strange stuff too... but I use it regularly in dense traffic it's great for stop and go traffic. the important thing is if you're driving you're driving. I don't turn it on and think oh sweet i can take a nap or read some hacker news posts... I keep my eyes on the road. There are bugs and I don't trust it but I do use it much like I use ChatGPT and the likes...

[MockObject]

How is it useful, if it requires such attention?

[kortex]

I have bog standard subaru EyeSight lane assist and dynamic cruise. It's quite nice, even though you are still "driving" and ready to take control. It reduces mental CPU by 50-80% (driving for me is practically like walking to begin with, largely subconscious). It's great in stop-and-go and long highway stretches.

[rootusrootus]

In my experience there is a pretty strong correlation between driving style and whether AP lowers your stress or increases it. If you are a very defensive driver, AP will frequently raise your blood pressure. YMMV, everyone has a different approach to driving.

[zitterbewegung]

IANAL but That’s not how class actions work. If the terms of service allow for the lawsuit then if the lawsuit rules in your favor you get a notification from the lawyers to accept or deny your share of the penalty.

There might be precedent to even removing the feature since FSD almost is vaporware and has no release date. https://www.cnet.com/tech/gaming/ps3-other-os-settlement-cla...

[ckwalsh]

No, I'm not an overly litigious person.

[maxdo]

it's a beta that you opt-in to use will all the warning and details. No chance.

[TaylorAlexander]

My Dad has a Tesla that he loves and I’ve told him “please don’t use the self driving features, they’re not well tested and have killed people.”

I understand the simpler lane keeping system is okay, but I don’t want to trust any system like this from Tesla given their track record with FSD.

[PUSH_AX]

Thanks for being a tester I guess.

This is the kind of feature I will use when the car and software in question has been battle tested for *years* with objectively excellent results.

[georgeg23]

I had the exact same issue, it was old M3 hardware.