| There is a lot of work going into hand tracking and natural interactions these days. People seem to feel much better and more immersed when they can use their hands to manipulate virtual objects. Oculus has this very neat trick with their controllers. Most of the buttons and sticks have some type of sensor that detects if you're touching it, even lightly. They use that information to alter the pose of the virtual hand. When the controllers are visible, your virtual fingers mostly match where your real ones are on the controller. At least one company builds unique user profiles that make your virtual hands the same size and skin tone as your physical hands, as well as adjusting for height and other parameters. Personally I find it gimmicky, but some people like it. Unfortunately, the technology isn't quite robust enough that we can fully get rid of controllers. Ultraleap is working on a system with multiple 3D IR cameras to track hands from multiple perspectives. I think this is the most likely path forward. There's some really crazy alternatives like strapping dozens of electrodes to the forearm to record muscle impulses that get fed to an ML model which extrapolates hand and finger movement. You're right, there's a lot of fundamental problems we have to work around, and it's far from easy. Even with Facebook's ridiculous resources, the oculus hand tracking is hot wet garbage for anything even slightly abnormal, like wearing a jacket. But, the industry as a whole is really picking up steam. It won't be too much longer before we solve most of the big problems. |