[collint]

Maybe you could use the wiimote in combination with the physical tracking to get more precise telemetry + precise physical location and AR.

[kreneskyp]

That was my thought a week ago. The kinect is limited in that it can only see the front of your body. Some movements obscure your body and it basically just freaks out.

[modeless]

You just described Playstation Move.

[NickPollard]

There have already been Playstation Move demos that do exactly this, with superb accuracy and great response times.

(Disclaimer: I work for SCEE, but this is my own opinion)