[gregmfoster]

I imagine this would be brutally hard for guitar - so often, my fingers are "touching" the correct strings, but not pushing down hard enough to trigger the note. Piano might be more feasible because you can watch the motion of the keys themselves.

Question - what is your root goal? Are you convinced video processing is the best way to achieve this?

[udit99]

Root goal is to allow this game (https://www.fretboardfly.com/) to be played via a "real life" instrument

[udit99]

So...instead of clicking/tapping on a screen, just play the guitar with the webcam on