[IntenseChaos]

The general consensus among my AV friends (who work at a bunch of different companies) is that their AV driving stack is really good, but obviously not perfect.

I have no idea about their business model and how COVID affects that, though.

[Fricken]

Relative to competitors Zoox's automous OS is doing quite well and doesn't get enough respect. Relative to the objective everybody is fucked.

[xiaolingxiao]

Could you provide more context on the first part.

[Fricken]

They've been keeping abreast.

The co-founder, Tim Kentley Klay was somehow able to get Jesse Levinson on board, and Jesse Levinson had no problem getting infinite street cred on board. So they were able to attract a lot of key, original robotics talent before the hype got out of control.

For a long time though, they were low on funds, so they did lots of closed course testing, and it wasn't until they closed a large funding round that Zoox began on public roads, and they performed quite well right out of the starting gate.

Now Zoox and it's competitors are lost in an endless wasteland of testing, development, and validation. It's futile to attempt to do a comprehensive analysis between the different players, they all have their quirks, but Zoox has built all the critical infrastructure needed to do full scale testing, and they're eyeballs deep in it like everyone else.

However, Zoox has stormy waters ahead financially. They need another $2 billion to stay abreast in this never ending race. It's getting harder to visualize scenarios where that happens.

What nobody can do well enough to build a competitive and scalable robotaxi service is prediction in multi-agent scenarios. The AI for that just doesn't exist.

[xiaolingxiao]

Multi-agent refers to behavior of pedestrians/cyclists, and other cars on the road. This is especially tough in "ambigous" junctions such as roundabouts, and unprotected left turns. There the strategy to negotiate the junctor is highly context dependent, and the information needed to find a strategy is not in the current scene. Drivers in these moments draw on "cultural awareness" of what "should" be done. Observing a history of what people do in these situations may not be sufficient because of the long tail of unique events, or at least unique in terms of how the computer will represent the scene. For example, if the scene is represented by the set of trajectories (or really waympoints), then the set of possibiilties is infinite. All of this assume the car "knows" it's entering and exiting a predefined scenario such as roundabout, real life driving is not so discrete.

On top of this, there's a liability and ethics issue. We accept teenagers for getting drunk and killing people, but we cannot accept an autonomous car that cannot navigate a roundabout which would otherwise be easy for a person, sober or otherwise.

[Fricken]

I have faith in robotaxis abilities to handle safety critical things. The lizard brain stuff is under control. They are still just too stupid to navigate complex traffic efficiently, without regularly hesitating and getting tripped up.

Robotaxis are Rube Goldberg machines, there are so many moving parts. The running joke at Waymo for a while was "How many engineers does it take to operate a self driving car?"

Everybody was convinced deep learning would give us all the magically brilliant AI we needed to make this work. With perception and classification problems the robotics industry was able to go from "impossibru" to "holy shit it works" over the space of a couple years, it was really exciting. In hindsight it's easy to see that the exciting and game changing breakthroughs were in fact a long time coming, and that the real rate of progress in open world robotics is in fact excrutiatingly slow and bespoke. Nobody has an ace up their sleeve.

[travisporter]

Can you elaborate on the moving parts? Is it just too expensive to install/maintain the sensors, or do you mean the algorithm has to deal with multiple inputs/decisions?

I could envision a scenario where a city council of a less populated city with lax regulations could deploy robotaxis to their economic advantage. Do you think we'll get there soon?

[xiaolingxiao]

Hm yeah sounds like what I have been hearing too. But the line engineer inside at Waymo are very optimistic at how close we are, maybe it's just the sentiment of the moment.

So in the scenario where predicting pedestrian/cyclist behavior holds up progress for a few more years. And given how the market has turned in SV and beyond, what's your read on how the space will play out? For example, car companies can't keep funding Aurora/Cruise/Argo because they will be facing very tough consumer climate, so the fight for funding internally will be even fiercer. Softbank funds Nuro and its portfolio of companies (WeWork and others) have been duds.

Google is expecting a bad 2020 ad revenue wise, unclear what will happen in 2021. The founders stepped out last year and the narrative has been that Google is less focused on "moonshots" and more on core ad business.

Is there any other deep pocketed investors that will finance development of AVs for another 5 years? Who will acquire the ones that are independent? IPO doesn't seem likely for any of them correct?

[ahnick]

What are some of examples of multi-agent scenarios they struggle with? Do you think there are paths to autonomous driving where we add infrastructure or laws to reduce the universe of these scenarios that would have to be dealt with? For example, adding dedicated autonomous driving lanes or reducing the amount of intersections between pedestrian walkways and roadways?

[matdehaast]

Thanks for the info. Can you clarify what predictions in multi agent scenarios are?

[Fricken]

Imagine an uncontrolled intersection. The Robotaxi is approaching from one direction. In the opposite direction is a cyclist who intends to turn left across the Robotaxi. There is also a pedestrian that may or may not cross the street, and another vehicle about to cross in front of the Robotaxi from the other direction. There are a huge number of ways this scenario can play out, and any decision made by one agent can affect the behaviors of all the others, compounding it's complexity. Humans can game out these situations intuitively, but current AI cannot read deep enough into the matrix to deal with these situations quickly and reliably.

[TheSmiddy]

In Australia there are no uncontrolled intersections (that I am aware of). Every single junction clearly marks who must give way and we don't have any 4-way stops, instead using roundabouts in these situations.

It's possible that for self driving to work road systems will have to be more formalised to remove the ambiguous situations you've described. I can't imagine it working well in China or Indonesia where traffic flows much more like water in a stream and lanes are merely just suggestions.

[nradov]

What is progress like on dealing with hand signals (or other gestures) from law enforcement or construction workers directing traffic?

[BubRoss]

There are rules to that situation. You can start there, let other cars go ahead of you if they break the rules, go slow and not hit anything. It isn't as if self driving cars can't look to the side or stop if something changes. Beyond that Jim Keller would say that not getting hit by something else is a matter of ballistics.

[ragebol]

I think was is meant is: in case you have a few people and AVs in an interaction, predict who's going to do what in order to best anticipate the overall outcome. Not sure humans can do that outside of conversation and norms and rules.