[cratermoon]

> if you had a driving sim wheel and 4 monitors for each direction + 3 smaller ones for rear view mirror, connected to a real world car with sufficiently high definition cameras, you could probably drive the car remotely as well as you could in real life, all because the images would map to the same latent space.

I disagree. When in a car, we are using more than our eyes. We have sound as well, of course, something that we provide feedback even in the quietest cars. We also have the ability to feel vibration, gravity and acceleration. Sitting in a sim without at least some of these additional forms of feedback would be a different skill.

[ActorNightly]

There was an even where they took the top iRacing sim driver and put him in a real F1 car and he was able to do VERY well in terms of lap times.

There was another even where they took another sim driver and put him in a real drift car, and he was able to drift very well.

Both vids are on youtube. Yes, real world driving has more variables, and yes, the racing drivers had force feedback wheels, but in general, if a person is able to control a car so well as to put the virtual wheel in the right square foot of the virtual track to take a corner optimally, its probably likely that most people could drive very well solely from visual feedback. Sound and IMUs can provide additional correctional information, but the key point still remains, is that whatever software runs has to deduce physics from visual images.

[et-al]

Would you say your examples are moving the sim driver from fewer sensors (an abstraction of driving) to more sensors (the real world)?

[cypress66]

Driving sims obviously have sound, and also feedback through the steering wheel (sometimes also the seat).

Self driving cars obviously have microphones and accelerometers too.

[pixl97]

https://youtu.be/HU6LfXNeQM4

I recommend watching this NOVA video on human perception. When doing any number of task, especially ones we do commonly we're using a ton of unconscious perception and prediction based upon our internal representation of physics and human modeling.

For example when I was younger I was noticing that I was commonly aware that a car was going to get over before it did so. I kept an eye out trying to determine why this was the case and I noticed two things. One is people commonly turn their head and check the mirrors before they even signal to get over. The other is they'll make a slight jerk of the wheel in the direction before making the lane change.