[venachescu]

I think you're missing the bigger point that he's trying to make - our sensory systems work in concert to provide a full picture of reality and they're very sensitive and extremely well tuned together through a lifetime of experiences.

Just providing an accurate projection of the visual field to the eye's doesn't solve the problem completely. Visual perception is not a passive mechanism, our eyes 'explore' the visual field through quick movements and readjustments and this process inherently locks together the proprioceptive signals from the ciliary and ocular muscles to the stream of information from the retina. By only providing screens, we can't simulate this process in a natural way and our body will detect it.

Similarly - our vestibular sense is strongly tied to our experiences of our body moving through the world. Just using a blanket stimulation of electrodes on the skin will not recreate these signals in anywhere near a convincing enough manner to fool the brain's perceptual processing.

All of our sensory perceptions are strongly linked together - if just one is perturbed in an unconvincing way (try wearing glasses that aren't prescribed to you for a while), our brain will do our "caveman brain" thing and make us sick.

[Houshalter]

On the other hand the brain is insanely adaptable. People made to wear glasses that flipped their vision upside down adjusted fine, and when they took them off, everything looked upside down.

Even your example is wrong. You can in fact adjust to glasses of the wrong prescription. People who need glasses have deformed eyes which is basically like having the wrong prescription. The brain adjusts, and people can go without glasses without vomiting everywhere.

The nausea reaction is an issue, but it doesn't affect the entire population. I have friends who can't play video games because of motion sickness, but no one argues that video games can't be commercially successful. And various tricks like removing head bob, adjusting the field of view, and adding a virtual "nose", have been shown to significantly alleviate that.

[the8472]

> our eyes 'explore' the visual field through quick movements and readjustments and this process inherently locks together the proprioceptive signals from the ciliary and ocular muscles to the stream of information from the retina. By only providing screens, we can't simulate this process in a natural way and our body will detect it.

That's the point of the light field displays GP mentioned. They also encode depth allowing the eyes to refocus on the correct depth corresponding to the object's virtual position.

Put differently, a LFD reconstructs photon paths, as if you were looking into a window facing the recorded object.

The remaining engineering problem is that those microlens arrays step down the effective resolution of the display, so you need much higher DPI to get the same sharpness as a non-LFD would get you.