[laura_g]

The literature has pretty consistently shown that adversarial examples can be found with only black box access (even with truncated prediction vectors), robustness methods are primarily a cat-and-mouse game between attackers and defenders, and the existence of adversarial examples is likely inevitable (https://arxiv.org/pdf/1809.02104.pdf).

The big question that remains is - so what? There's exceedingly few use cases where the existence of adversarial examples causes a security threat. There's a lot of research value in understanding adversarial examples and what that tells us about how models learn, generalize, and retain information, but I am not convinced that these attacks pose a threat remotely close to the amount of attention given.

[owlbite]

Self driving cars seem like a dangerous threat vector if an adversarial image can be deployed in such a way as to cause them to commit dangerous maneuvers on demand.

[laura_g]

I completely agree, but that's a very big "if". I'm not terribly familiar with autonomous vehicle driving systems, but my passing understanding is that there are multiple components working together that help make predictions, and these systems do not rely on any single point of failure.

The classic example of a sticker on a stop sign is, in my view, more of a dramatization than a real threat surface. Designing an adversarial perturbation on a sticker that can cause misclassifications from particular angles and lighting conditions is possible, but that alone won't cause a vehicle to ignore traffic situations, pedestrians, and other contextual information.

Plus, if I wanted to trick a self driving vehicle into not stopping at an intersection, it would be much easier and cheaper for me to just take the stop sign down :)

[Isinlor]

There is plenty of natural "adversarial examples" to worry about.

Like billboard with stop sign on it.

https://youtu.be/-OdOmU58zOw?t=149

[genewitch]

I'll be more inclined to start believing that self driving / autonomous vehicles are actually "coming soon" when the federal government decrees it is illegal to wear clothing with certain markings/colors. No red octogons, no reflective red and white parts, no yellow vertical stripes, etc.

I don't think that "cause an air to fail to stop" is the correct threat to address, I think "making AI stop and therefore cause traffic" is.

Wake me up when I can have any two arbitrary addresses as start and end points and a machine or computer can drive me between them, 24/7/365 - barring road closures or whatever.

[Isinlor]

My prediction is that it will happen with 50% confidence before/after 2029. Or 50% confidence that it will be between 2026 and 2031.

Basically they need to improve their driving software some 10 000x times. From driving 100km before safety critical disengagement to 1 million kilometers. 1 - 2 million milles is benchmark presented by CJ Moore, Tesla’s director of autopilot software to California Department of Motor Vehicles.

> “Tesla is at Level 2 currently. The ratio of driver interaction would need to be in the magnitude of 1 or 2 million miles per driver interaction to move into higher levels of automation. Tesla indicated that Elon is extrapolating on the rates of improvement when speaking about L5 capabilities. Tesla couldn’t say if the rate of improvement would make it to L5 by end of calendar year.”

If they manage to keep on doubling distance driven every 6 months then we should be there in:

log2(10000) * 6 months = 8 years

You can make your own predictions here: https://www.metaculus.com/questions/5304/widely-available-te...

[nerdponx]

If I really had to choose, would rather have freedom of expression than AI cars. But maybe that's just me.