[hodwik]

But 'all' you have to do is track the pupil, and push focus on the object they're looking at. We're already rendering the focal length live in most games, there's no reason we couldn't shift the focal length affording to what the viewer's pupils are pointed at.

Alternatively, they could use a modified biology, where the direction the head is pointing chooses the relative DOF, basically you render a fairly wide depth of field around whatever the head is pointed at.

Anyway, you can get a pretty lifelike experience with just some basic atmospheric perspective. Look at photography with very wide depth of field [1], so everything is in focus simultaneously. You sorta just get used to it, and accept it.

[1] http://cdn.shutterbug.com/images/0814picture03.jpg

[andrewstuart2]

Except you can't fake the feedback of muscles adjusting focus that the real world requires without different physical focal planes. You can mimic the focal effect, but your brain has the ability to recognize motor inputs to the eyes and act on this information (or lack thereof).

I think what we really need is a way to target specific focal distances via some sort of dynamic refraction. Otherwise we'll never get the proper motor feedback needed for immersion.

[Natanael_L]

Light field displays seems to be the answer. Also much more computationally heavy.

[nitrogen]

So, dynamic lenses on the VR display.

[modeless]

As someone who works on eye tracking for VR headsets, this is much easier said than done. Doing it well requires levels of accuracy, latency, and reliability that are not available in any eye tracker on the market today.

[rjvs]

What about SMI already tracking pupils at 250Hz? https://www.youtube.com/watch?v=Qq09BTmjzRs That tech is still expensive and I don't know how accurate or reliable but if it is good enough for foveated rendering, I would have expected it to be good enough for focus as well. Care to elaborate as to why it wouldn't work, or is it just beyond your horizon of "today"?

[modeless]

It's not accurate or reliable enough (in addition to being very far from a consumer price point). Reliability in particular is difficult to assess and usually lacking in eye tracking systems. Also just because the cameras are running at 250 Hz doesn't mean the latency is 1/250 seconds or even close to that.

[rasz_pl]

Only reason for lack of availability is size of the market. Tech is already here and you are probably holding it in your hand right now. Avago's will be more than happy to sell you tiny combo chip packing camera/DSP able to capture >10K frames/s, track at >5 meters/s and report position at over 1KHz.

[chimprich]

As someone working in the industry, would you care to guess how many years away we are from being able to do this at an acceptable level? Is this something for the next gen of hardware or is it much further away?

[Animats]

Maybe VR would work better if you just fed the same view to both eyes and ignored stereoscopy. Jaron Lanier used to do that with his original VR rig in the 1980s when one of the machines went down. He said nobody noticed.

[milesokeefe]

I can assure you that the tech has advanced enough in the last three decades that you would immediately notice the lack of non-parallax depth perception.

[provemewrong]

The problem is your eyes can focus at different focal planes: on an object, past an object etc. I'd imagine it would be pretty jarring if the software just blurred everything around an object that happened to be at the x, y coordinates where your pupils were pointing.

[jsprogrammer]

You project into the scene to discover the distance of the object, then set the aperture and/or focal length of your camera as appropriate.

[ascorbic]

This doesn't help. You need to physically change focus in your eye. Your brain uses the feedback from the eye muscles to ascertain distance. If your eye muscles don't need to adjust focus, then they're reporting that everything is at the same distance. If this disagrees with the stereoscopy or the context then you will get nausea.

[hodwik]

> "Your brain uses the feedback from the eye muscles to ascertain distance."

Source? I'm under the impression that focus is a brain-first, muscles-second process. If the object was already in focus, I don't think your eyes would mind.

[ascorbic]

It's called the vergence-accommodation conflict. There's quite a bit of literature on the problem in relation to VR. Here's one paper picked at random from Google. http://jov.arvojournals.org/article.aspx?articleid=2122611

[hodwik]

If that were true people with only one eye would be nauseous all of the time.

As I understand it (which I may not) -- this paper (which purports to be the only paper to have tested this effect), says that they test divergence by having the images displayed at various distances to replicate varying levels of divergence.

They see that the closer it is to a realistic vergence/accommodation match the more comfortable the eyes.

However -- it should obviously be easier to tune a 3D image which is at a realistic distance than one which is not.

It seems to me, they may very well just be showing that effect, just showing that it is easier to adjust the illusion to render realistically at distance. This would make perfect sense.

This would then just argue for better tuning.

[Natanael_L]

Light field displays?

[marvin]

What do you do when the pupil is ambiguously pointing at an object 30cm in front of your eyes, and a mountain on the horizon?

[hodwik]

Wide DOF