[unionpivo]

Just because humans can generally do it without lidar and other sensors, does not mean that lidar and other sensors, would not improve human driving, if we could somehow integrate them seamlessly (and well reduce their price).

A car equipped with more sensors, will always be better at handling more situations, than the one without them, no matter how "smart" their drivers become.

[fastball]

I think the overlooked part is "seamlessly integrate". Signal to noise ratio is important, and it is not trivial to "just add sensors". I think the analogy of "too many cooks in the kitchen" applies here.

[ra7]

I think the keyword you’re looking for is ‘sensor fusion’, which apparently is only a problem for Tesla because the others have figured it out years ago. Perhaps there is a talent gap at Tesla or an unwillingness to invest in it. They generally avoid having to do anything that’s hard and try to look for shortcuts.

Case in point: the whole radar removal and reintroduction flip flop. It seems like they don’t believe themselves that cameras are sufficient.

[fastball]

I don't think anyone has "figured it out" if they still don't have unconstrained driverless cars.

[ra7]

That’s tangential to your original point of discussion which was sensor integration. Sensor fusion is absolutely a solved problem. Autonomous driving requires more than just sensor fusion though.

[fastball]

It's not tangential at all. If the purpose of these sensors is for autonomous driving, and "solving" sensor fusion hasn't gotten you there, how can you possibly say it is a solved problem? Sensor fusion (in this context) is not solved until you are using fused sensors for autonomous driving.

[ra7]

Why are you equating sensor fusion with solving the entirety of autonomous driving? That’s disingenuous. You understand sub problems can be solved, right? Sensor fusion is just combining different sensor inputs so you have high confidence in your object detection and you’re not “confused with disagreements” as Tesla likes to say. Self driving involves solving prediction and planning problems too, not just sensor fusion.

[surfpel]

Indeed. Ultimately humans do it remarkably well with just vision, too, when they’re not intoxicated, distracted, or falling asleep.

[gonehome]

Yeah, adding sensors can sometimes make things worse.

[xnx]

Example? There are plenty of examples where mistakes that Tesla's cameras make (being blinded by the sun, having trouble with emergency vehicles) are no problem for systems with more sensors.

[gonehome]

This talk was worth watching (any Karpathy or Carmack talk is imo): https://www.youtube.com/watch?v=g6bOwQdCJrc

Specifically this is the relevant part: https://youtu.be/g6bOwQdCJrc?t=1370

They removed radar because it was making things worse.

[xnx]

I interpreted that statement by Karpathy as an accidental admission of Tesla's own limitations and not of the approach itself. (Note: I'm upvoting all your comments because they're very reasonable and supported.)

[stevenhuang]

> A car equipped with more sensors, will always be better at handling more situations

Yes, integrating seamlessly is the hard part.

For Tesla, they found the increased sensor fusion complexity made the overall system less reliable, which was what informed their attempt at vision only. Karpathy went into this on Lex Fridman's podcast.

Only with better sensor fusion algorithms that worst case the overall system performance doesn't degrade with additional sensors. Tesla could have hit compute bounds in order to meet latency requirements for example (or they couldn't find the right technique/algorithms).

Sounds like a hard problem but I imagine this would eventually be overcome in the future. More sensors is definitely the future.

[xnx]

This is retroactive justification for using low-cost sensors. There's a chance they get lucky and pull it off with only cameras, but the chance of Tesla doing that is much less likely than Waymo getting their sooner with more sensors (or even even the Waymo driver achieving it with only vision if they chose. Using a full suite of sensors gives Waymo much better training data if they ever chose to use vision-only.

[gonehome]

I think Tesla does use Lidar in training for what it's worth (at least I recall someone telling me this - I think it was a friend that worked there but not 100% sure I'm remembering correctly).

It's possible the lidar approach only really works with up to date high resolution maps and without that you end up in a local max you can't really escape with that approach.

That's been Tesla's argument up to this point anyway and I don't think the success in cities really proves things either way.

It may not matter since robotaxis in cities is still worth a lot and highway driving is mostly a solved problem.

True full self driving though will probably require solving vision - in that case I think the argument that lidar is a local max could very well be true.

[ra7]

> It's possible the lidar approach only really works with up to date high resolution maps and without that you end up in a local max you can't really escape with that approach.

Sorry, but you’re just repeating Elon Musk’s buzzwords like “local maximum”. It doesn’t make sense. There’s no such thing as “lidar approach”. All sensor inputs are fused (early/late fusion) and run through perception algorithms. Lidar is used for localization with maps, but that’s not even its primary use (that’s object detection). And it does not require maps to be up to date either.

[xnx]

I expect Tesla's rationalization will suddenly change when the cost for lidar units drops to hundreds of dollars. At that point, Waymo will have a huge head start.

[ra7]

Lidar cost has already dropped a lot. Waymo’s lidar on their I-Pace is 90% cheaper than the previous gen. We’re also seeing car manufacturers (GM, Volvo/Polestar, Mercedes, Chinese OEMs) include inexpensive lidar units for driver assistance.

Tesla’s problem is that they’ve promised their existing cars on the road are capable of full self driving. They can’t add hardware to them, so they will maintain the “camera is sufficient” stance for a long time.