Oh nice, yes that's exactly what I'm doing, I take the middle of my hand, take the HSR point and increase it's range to get the whole hand. I'm listening :)
Nice! You can implement a real time skin segmentation algorithm using HSV segmented skin as training, and using random forests to do classification, it classifies in real-time with opencv!
Here is a video of my result from only training with one frame of a segmented HSV hand to some random videos: https://vimeo.com/114489035
Also, you can use a kinect or an occipital structure sensor to do segmentation from a certain depth, more hardware but less vision computation.
I was working on skin segmentation for a research project to it segment in real time for Google Glass! I never did the training with a huge database of ground truths. I want to eventually complete the training right to build a robust skin segmentation alg. I can link the github page(Opencv implementation only), but my code is really messy as I built as a POC in two days.
The google glass app was to use skin segmentation for gesture recognition.
We had a student once who used a face detector to first find the face area, and then extracted the skin colour from that region for the subsequent hand tracking.
Here is a video of my result from only training with one frame of a segmented HSV hand to some random videos: https://vimeo.com/114489035
Also, you can use a kinect or an occipital structure sensor to do segmentation from a certain depth, more hardware but less vision computation.