[dsf2d]

Its not ever going to get ready.

Getting this to a place where it is better than humans continuously is not equivalent to fixing bugs in the context of the production of software used on phones etc.

When you are dealing with a dynamic uncontained environment it is much more difficult.

[SpicyLemonZest]

Waymo is in a place where it's better than humans continuously. If Tesla is not, that's on them, either because their engineers are not as good or because they're forced to follow Elon's camera-only mandate.

[moralestapia]

It's the camera-only mandate, and it's not Elon's but Karpathy's.

Any engineering student can understand why LIDAR+Radar+RGB is better than just a single camera; and any person moderately aware of tech can realize that digital cameras are nowhere as good as the human eye.

But yeah, he's a genius or something.

[cameldrv]

Digital cameras are much worse than the human eye, especially when it comes to dynamic range, but I don't think that's all that widely known actually. There are also better and worse digital cameras, and the ones on a Waymo are very good, and the ones on a Tesla aren't that great, and that makes a huge difference.

Beyond even the cameras themselves, humans can move their head around, use sun visors, put on sunglasses, etc to deal with driving into the sun, but AVs don't have these capabilities yet.

[CydeWeys]

> especially when it comes to dynamic range

You can solve this by having multiple cameras for each vantage point, with different sensors and lenses that are optimized for different light levels. Tesla isn't doing this mind you, but with the use of multiple cameras, it should be easy enough to exceed the dynamic range of the human eye so long as you are auto-selecting whichever camera is getting you the correct exposure at any given point.

[tzs]

Tesla claims that their cameras use "photon counting" and that this lets them see well in the dark, in fog, in heavy rain, and when facing bright lights like the sun.

Photon counting is a real thing [1] but that's not what Tesla claims to be doing.

I cannot tell if what they are doing is something actually effective that they should have called something other than "photon counting" or just the usual Musk exaggerations. Anyone here familiar with the relevant fields who can say which it is?

Here's what they claim, as summarized by whatever it is Google uses for their "AI Overview".

> Tesla photon counting is an advanced, raw-data approach to camera imaging for Autopilot and Full Self-Driving (FSD), where sensors detect and count individual light particles (photons) rather than processing aggregate image intensity. By removing traditional image processing filters and directly passing raw pixel data to neural networks, Tesla improves dynamic range, enabling better vision in low light and high-contrast scenarios.

It says these are the key aspects:

> Direct Data Processing: Instead of relying on image signal processors (ISPs) to create a human-friendly picture, Tesla feeds raw sensor data directly into the neural network, allowing the system to detect subtle light variations and near-IR (infrared) light.

> Improved Dynamic Range: This approach allows the system to see in the dark exceptionally well by not losing information to standard image compression or exposure adjustments.

> Increased Sensitivity: By operating at the single-photon level, the system achieves a higher signal-to-noise ratio, effectively "seeing in the dark".

> Elimination of Exposure Limitations: The technique helps mitigate issues like sun glare, allowing for better visibility in extreme lighting conditions.

> Neural Network Training: The raw, unfiltered data is used to train Tesla's neural networks, allowing for more robust, high-fidelity perception in complex, real-world driving environments.

[1] https://en.wikipedia.org/wiki/Photon_counting

[iknowstuff]

all the sensor has to do is keep count of how many times a pixel got hit by a photon in the span of e.g. 1/24th of a second (long exposure) and 1/10000th of a second (short exposure). Those two values per pixel yield an incredible dynamic range and can be fed straight into the neural net.

[iknowstuff]

https://www.sony-semicon.com/files/62/pdf/p-15_IMX490.pdf

The IMX490 has a dynamic range of 140dB when spitting out actual images. The neural net could easily be trained on multiexposure to account for both extremely low and extremely high light. They are not trying to create SDR images.

Please lets stop with the dynamic range bullshit. Point your phone at the sun when you're blinded in your car next time. Or use night mode. Both see better than you.

[epistasis]

I have enjoyed Karpathy's educational materials over the years, but somehow missed that he was involved with Tesla to this degree. This was a very insightful comment from 9 years ago on the topic:

> What this really reflects is that Tesla has painted itself into a corner. They've shipped vehicles with a weak sensor suite that's claimed to be sufficient to support self-driving, leaving the software for later. Tesla, unlike everybody else who's serious, doesn't have a LIDAR.

> Now, it's "later", their software demos are about where Google was in 2010, and Tesla has a big problem. This is a really hard problem to do with cameras alone. Deep learning is useful, but it's not magic, and it's not strong AI. No wonder their head of automatic driving quit. Karpathy may bail in a few months, once he realizes he's joined a death march.

> ...

https://news.ycombinator.com/item?id=14600924

Karpathy left in 2022. Turns out that the commenter, Animats, is John Nagle!

[xiphias2]

Using only cameras is a business decision, not tech decision: will camera + NN be good enough before LIDAR+Radar+RGB+NN can scale up.

For me it looks like they will reach parity at about the same time, so camera only is not totally stupid. What's stupid is forcing robotaxi on the road before the technology is ready.

[moralestapia]

>reach parity at about the same time

Nah, Waymo is much safer than Tesla today, while Tesla has way-mo* data to train on and much more compute capacity in their hands. They're in a dead end.

Camera-only was a massive mistake. They'll never admit to that because there's now millions of cars out there that will be perceived as defective if they do. This is the decision that will sink Tesla to the ground, you'll see. But hail Karpathy, yeah.

* Sorry, I couldn't resist.

[algo_trader]

Was Karpathy "fired" from Tesla because he could not make camera only work ?

Or did he "resign" since Elon insists on camera-only and Karpathy says i cant do it?

[wstrange]

Clearly they have not reached parity, as evidenced by the crash rate of Tesla.

It's far from clear that the current HW4 + sensor suite will ever be sufficient for L4.

[aeternum]

If you ride enough Waymo, you will realize it is a far more cautious driver than a human but not a better driver. If you need to get somewhere even at an average speed, you still take uber/lyft.

Waymo still takes many wrong turns and can easily get stuck in situations where a human would not.

[bluGill]

citation needed. Waymo says they are better, but it is really hard to find someone without a conflict of interest who we can believe has and understands the data.

[SpicyLemonZest]

I reject the premise of your comment. If Tesla wants to convince people that Robotaxi is safe, it's on them to publish an analysis with comparative data and stop redacting the crash details that Waymo freely provides. Until they do, it's reasonable to follow the source article's simple math and unreasonable to declare that there's no way to be sure because there might be some unknown factor it's not accounting for.

[bluGill]

That is a first step - but anything published by tesla or someone they pay is suspect and so independant annalisys is mandatory. That includes some data gathering to ensure the data tesla/waymo gives is correct.

[xiphias2]

It's clear that camera-only driving is getting better as we have better image understanding models every year. So there will be a point when camera based systems without lidars will get better than human drivers.

Technology is just not there yet, and Elon is impatient.

[MBCook]

Then stop deploying camera only systems until that time comes.

Waymo could be working on camera only. I don’t know. But it’s not controlling the car. And until such a time they can prove with their data that it is just as safe, that seems like a very smart decision.

Tesla is not taking such a cautious approach. And they’re doing it on public roads. That’s the problem.

[fwip]

> So there will be a point when camera based systems without lidars will get better than human drivers.

No reason to assume that. A toddler that is increasing in walk speed every month will never be able to outrun a cheetah.

[shoo]

in contrast, a toddler equipped with an ion thruster & a modest quantity of xeon propellant could achieve enough delta-v to attain cheetah-escape velocity, provided the initial trajectory during the first 31 hours of the mission was through a low-cheetah-density environment

[tzs]

That initial trajectory also needs to go through a low air density environment. At normal air density near the surface of the Earth that ion thruster could only get a toddler up to ~10 km/h before the drag force from the air equals the thrust from the ion thruster.

The only way that ion thruster might save the toddler is if it was used to blast the cheetah in the face. It would take a pretty long time to actually cause enough damage to force the cheetah to stop, but it might be annoying enough and/or unusual enough to get it to decide to leave.

[shoo]

> low air density environment. At normal air density near the surface of the Earth that ion thruster could only get a toddler up to ~10 km/h

agreed. this also provides an explanation for the otherwise surprising fact that prey animals in the savannah have never been observed to naturally evolve ion thrusters.

[sschueller]

Lidar and radar will also get better and having all possible sensors will always out perform camera only.