Okay how about we make cars that can fully self-drive, and then remove hardware crutches afterwards. Some blind people can ride a bike safely (with echolocation), but I'd strongly recommend you use eyes too if you have them.
Humans have millions of years of evolution tuning our visual system and we still have tons of crashes due to poor performance during low light situations, high contrast, etc. Twilight is recognized as a time of increased accidents, we’ve had decades of mandating better types and positions of lights, roads are expensively designed with various paints and reflectors, etc.
Not a computer vision engineer here - do we just not know how the human brain recognizes objects or does it do so in a way that's hard to reproduce in silicon?
We don't know how it works, and the parts that we do know are very hard to implement.
In particular, our vision is very very tied to our model of the world, and is clearly an evolutionarily derived trait, not something we learn from scratch as children. We are basically born with a model that can augment the raw sensory input with basic physics (object permanence, but also we have an intuition for things like "heavy = large momentum even at slow speeds", and for "what goes up will come down", some basic optics of how objects cast shadows, and others). In addition, we learn to recognize objects and we know their actual sizes from past experience, so we typically estimate distance this way far better than relying on parallax. "Small image of car" is automatically "car far away", even if seen with one eye, because we know what size a car is.
In fact, much of our brilliant vision starts going awry immediately in an artifical setting with misleading sized objects.
We don't have much idea of what "our model of the world" is. Probably quite little of it is genetically encoded. E.g. object permanence has to be learned (takes around 7 months).
What you describe is largely the classical "cognitivist" model, which is more or less dead in the contemporary tought. We don't even have a very good account of depth estimation, apart from "bunch of things seem to affect it".
Object permanence is not "learned", it develops at around 7 months. Same as you don't "learn" to reach puberty, your body is pre-built to only start puberty at a certain age. Edit: I should note that object permanence in particular is a more contentious topic. Other parts of of our understanding of the world are more readily assigned to innate functions, though.
Note that many mammals have object permanence, and for some it develops right after birth. The fact that human development is so slow is the only reason some of these things look like actual learning. Another more clear example is our ability to walk/crawl - in humans it seems to be learned, but most other mammals are able to do it within minutes of being born, making it quite obvious it is in fact a genetically coded trait.
If the babies don't see objects, they don't "develop" object permanence. Yeah, here we can summon the "use it or use it" copeout, but then the nativist account becomes unfalsifiable.
Being able to crawl within minutes (and even somewhat run within hours as horses do) is mostly due to anatomy and "reflex pathways". E.g. a cat can walk without brains [1], as can famosly headless chicken.
Human babies too have these kinds of reflexes but they are "abandoned" well before the baby learns to walk.
While I agree it makes it very hard to test some of these hypotheses, "use it or lose it" is very clearly true of at least some bodily and cognitive capacities. So it's not unfair to posit it could apply to these as well.
If you think quadruped walking is only a bunch of reflexes and human walking isn't, that still doesn't completely negate my point. Primate infants also display significantly more agility than human babies right afer birth, and "learn" to walk significantly faster.
Here [0] is a study for example that finds that the age of walking is 94% predictable for any mammalian species that walks on the ground by a particular ratio between the mass of the infant brain and the adult brain (that is, an infant animal essentially learns to walk when its brain reaches a certain percentage of its final brain size).
Besides this type of animal comparison, another signal we can use to distinguish between built-in capacities and things we learn from experience is to see whether a capacity can be developed later in life. For example, if you lack visual stimuli in the early part of brain development, even if you get it later, you will never develop vision (we know this from studies on kittens...).
Cognitive scientist here: we have no idea how the brain works in general. In locomotive tasks like driving the brain probably does not even try to recognize objects most of the time.