[carapace]

It really really bothers me that these folks are using a live city with real, non-volunteer test subjects of all ages (little kids and old folks use public streets) as a test bed for their massive car-shaped robots.

It's bad enough that people are driving cars all over the place, car collisions have killed more Americans than all the wars we've fought put together.

I'm one of those people who say, "Self-driving cars can't happen soon enough." But I don't think that justifies e.g. killing Elaine Herzberg.

Ask yourself this, why start with cars? Why not make a self-driving golf cart? Make it out of nerf (soft foam) and program it to never go so fast that it can't brake in time to prevent collision.

Testing these heavy, fast, buggy robots in crowds of people is extremely irresponsible.

[torpfactory]

There is a different perspective that you could use (and I’m not necessarily advocating for it; hear me out):

Human driven cars are dangerous to the tune of ~36,000 deaths per year. Every year without the implementation of full self driving we pay some large percentage of that number in lives. Self driving cars won’t make it out of the lab without real driving on real roads in real scenarios. Taking appropriate precautions (a human safety driver, maybe two) and testing in the real world might save more lives overall than keeping the vehicles in a more lab-like setting for longer, and missing some of the complexity of the real thing.

[carapace]

What about the nerf-golf-cart idea?

[pj_mukh]

Most of these companies run closed-circuit tests (aka nerf-golf-carts) and 1000's of hours of simulation. In the end, the only test that matters is the real-test. Real speed (sensor/computation latency et al.), real sensor feeds (lighting et al.) and real car size (momentum et al.). Source: Am Controls engineer.

There WILL be bugs and un-modellable sources of error. The real hedging in these situations is the safety driver. The death of Elaine Herzberg is very regrettable, but the fault ultimately lies with the safety driver and the training that was offered to her. She was on her phone, like 1000's of drivers are now.

[carapace]

> Most of these companies run closed-circuit tests (aka nerf-golf-carts) and 1000's of hours of simulation.

I don't think we're talking about the same thing.

I mean build a machine that, in the real world, can't hurt people.

Make it light.

Make it soft.

Program it to limit its speed such that it can always stop before colliding with whatever (whoever) might leap out in front of it.

If the top speed is five miles per hour, so be it.

The safety driver is wrong too. But she was there because Uber wanted to put car-shaped robots onto public streets.

Really the insane thing is that we mix car and pedestrian traffic at all in the first place. Oddly enough, it's the result of a deliberate campaign of propaganda: https://www.youtube.com/watch?v=-AFn7MiJz_s "Adam Ruins Everything - Why Jaywalking Is a Crime"

> Adam reveals the derogatory origins of jaywalking and explains how the auto industry made it illegal.

[pj_mukh]

"If the top speed is five miles per hour, so be it."

You are not really testing though. The whole point is to build a machine that can go the speed limit. You can test all you want at 5mph, but let me assure you, most of the real issues will show up when you go 45 in the real world.

[carapace]

> The whole point is to build a machine that can go the speed limit.

Sure, eventually, when the machines and sensors and algorithms and so on are good enough. When the infrastructure can be rebuilt to accommodate them (e.g. Boring company tunnels for auto-trucking, sensors and comms in the roads and signage, &c.)

The rush to market is the whole problem. Not the ultimate concept.

I want a machine that can take my mother to her doctors appointment now that the dementia has gotten to the point where she shouldn't take the bus on her own anymore.

Let the experiences with the toy cars guide the incremental graceful adoption of faster machines.

> most of the real issues will show up when you go 45 in the real world.

Right! So don't go 45.

[tom-_-]

I imagine reducing the weight of the vehicle would effect it's stopping distance, traction, turning radius, a bunch of other car physics that would in turn produce unreliable data to train on.

I assume much of the training is done in simulation or within a controlled environment but unfortunately the only way to train for city driving is to gather as much real world data as possible and that means "testing in production" with a hopefully alert humans (one for backup) behind the wheel.

[torpfactory]

Speed is an important parameter: you can’t do freeway driving in a golf cart. IMO regular vehicles can be safe enough with appropriate supervision. It’s really once you get to the first runs without safety drivers where your backup goes away. Hopefully that will be only after your intervention rate is zero.

[carapace]

We can work our way to that point (freeway driving) without killing people.

Really, the problem is the rush to market not the idea itself.

[torpfactory]

That’s the heart of the original argument I posed on this thread: we really should rush to market because the status quo is quite unsafe.

[carapace]

The status quo is insane, IMO. I call our mixed ped/car transportation networks the "mayhem lottery" (As in, every time you go out there you're taking a chance that you won't come back with all your limbs, or your life.) It was years ago, but a neighbors son was crossing the street in a crosswalk when someone ran a stop sign and knocked him fifty feet. I've twice seen little old ladies lying dead in the street from hit and run drivers.

Making bad imitations of KITT from Knight Rider is not the solution here.

https://en.wikipedia.org/wiki/Traffic_calming is cheaper and easier, for example.

Look I want robot cars, okay? I even want them ASAP.

As soon as possible without testing robots on public ways.

Build a fake city, populate it with people who have signed waivers, test there.

Sure it's expensive but at least you don't risk killing more innocent people with your experimental car-shaped robots.

I mean, what if I built a television with H.E.M? (Human Eradication Mode) If it escapes and kills someone, isn't that my fault?

Maybe if I put a human in there, give her a phone to distract her, and call her "Safety Driver", I can deflect any blame for my robot's killing spree onto her and get away with murder scott free! It's the perfect crime!

Okay, sorry, I got a little carried away there. But I hope my point is clear: Safety: Yes. Robot Cars: also yes, but obeying the First Law of Robotics. Testing robots cars on innocent people: hard no.

[anigbrowl]

In the video, which was recorded with a safety driver behind the wheel

I can live with it. Human drivers annoy me so much that throwing the dice on autonomous cars is not a big stressor to me.

[KKKKkkkk1]

I think you're missing the narrative that the self-driving industry is pushing here. They "solved the problem" and their fleets driving around "autonomously" is being done in order to demonstrate this to the public. A golf cart is obviously unsuitable for that purpose.

I think this narrative has run out of steam at this point, by the way. Waymo's valuation has gone from $175B to $105B to $30B since 2018. Zoox specifically is now laying off engineers.

[reitzensteinm]

Waymo has had a single round with outside money, and that valued it at $30bn.

[yalogin]

Wait, Zoox was valued at 175B at one point? Is this true? Can you point me to any resources on this?

[Xorlev]

GP said:

> Waymo's valuation has gone from $175B to $105B to $30B since 2018. Zoox specifically is now laying off engineers.

Waymo.

[z3rgl1ng]

They said Waymo had that valuation.

[AndrewKemendo]

You can't learn to operate in environments you don't train in. It would be great if we had a solution to the out-of-distribution inference/reward problem, but I don't think it really exists.

I'm firmly in the "Perfect for freight, questionable value for consumers" camp WRT autonomous cars. I also think it's irresponsible to do this but the reality is, they are doing all the socially "appropriate" things, like get approval from the city.

[carapace]

It's one of those things where I wonder if I'm not being too much of a curmudgeon. I'm sure the case could be made that these things will reduce overall traffic deaths long before they become perfect drivers, eh

[AndrewKemendo]

Yeah I don't buy that argument. These systems would have to be at least human level and also so pervasive that it would be almost like an inoculation.

At that point there are cheaper and easier ways to do that, which by the way are already happening. If you buy a modern car they have very impressive look ahead/smart cruise and lane keeping systems.

[carapace]

I agree with you, I think. To me it seems that you pretty much need AGI to drive IRL. And there are a lot of cheaper and easier things to do in the meantime. Cheers

[baby]

> It's bad enough that people are driving cars all over the place, car collisions have killed more Americans than all the wars we've fought put together.

Do you want this to stop? Then we're going to have this people test their self-driving cars in a real environment. The more we delay this, the more people are dying because of car accidents.

[Dumblydorr]

Those were already made years ago at the start of SDC innovation. A few companies are way beyond the worst human drivers now, there's already a massive amount of motor vehicle death caused by intoxication that we should worry about. Not fantasy robodeaths that we can count on one hand.

[def8cefe]

> fantasy robodeaths

They're not a fantasy, they've occurred. The reason we can count them on one hand is because few cars currently drive autonomously and there is a fail-safe human at the wheel who (most of the time) is paying attention to the road.

Even if autonomous cars are better than human drivers they will still inevitably strike and kill pedestrians and vehicle occupants; they are not a magical solution to vehicle collisions.

[wtvanhest]

I used to live in north beach on grant and Union, and these cars were driving by constantly. I knew right away that they would show it there.

I forgot how annoying it was.

[lonelappde]

Elaine Herzberg was killed by a human driver not watching the road.

[carapace]

My understanding is that the software in the car detected Herzberg and could have stopped the car in time to avoid the collision but that that subroutine had been disabled due to to many false positives. The "safety driver" (in quotes because she turned out to be both unsafe and not actually driving at the time) is also at fault.

Certainly, Elaine Herzberg wouldn't have been killed by that car if it wasn't there, eh?

Don't test killer robots on the public.