Have you tried it? Not a great experience. Apps don't get access to the camera feed so they can't actually detect your keyboard, you have to do a manual registration process which is janky. But the bigger problem is the quality of the passthrough. Depth perception and latency are especially problematic for this application. Cool demo but not worth using in practice. Maybe it could be better on Vision Pro.
Well, the point was the suggested software already exists, not that it would get the approval of everybody.
I agree that formal training and teaching with a quality provider is likely to be optimal but any tool that adds to the available options for training is worth exploring, especially if somebody already has access to the platform the software is available for - paid instruction is not always an option for many.
We also need to be careful not to gatekeep music by demanding instrument players go a formal route of instruction.
I'm working on AI piano lessons right now (extremely rough landing page at https://trebel.la/ while I work on the core tech). I am drooling at the idea of integrating with this thing one day...
My teacher literally would spend one hour with me on one page of sheet music to ensure I can play the notes evenly with the correct dynamics. He could hear the tiniest hesitation I have, and then come up with specific methods to help me practice that part. That is why I don't think any of such software, AI or not, actually knows how to teach piano, and I doubt they ever will.
I tend to agree with you, as I have spent many years in lessons myself and taught for a couple. But a) real lessons are expensive and difficult to access for a lot of people, and b) even for someone who has a teacher, most practice time is self-directed. I think there's a real opportunity to make practice much more efficient, and also provide a better experience than currently exists for those who can't access a human teacher.
Also, multimodal LLMs are improving so quickly that I wouldn't be that surprised if they were able to do what you're describing in a couple of years. But they are definitely not there yet.
You may want to look to Sight reading factory. They do a pretty good job allowing you to dynamically generate unlimited sheet music according to your pianistic capability.
Have been taking piano lessons for almost a decade now (very committed), I can tell you most piano teachers are not going to care about this kind of gimmicks. Most teachers don't even want to bother with online lessons over Zoom or whatever -- you either go to their home for lessons on a real acoustic piano or you don't have lessons. Only the most eccentric teacher is going to spend $3.5K on this toy and expect their student to do so as well.
Music teachers don't dislike Zoom because they're reluctant to use tech in the abstract, they avoid it because trying to teach a music lesson with a bad mic, a half-second delay, and a low-FOV camera pointed haphazardly is a nightmare.
Source: many family members who are music teachers, spanning multiple generations and a wide range of technical prowess.
This would still have tactile feedback because you would still be playing a piano. Imagine things like scales being overlaid in a different color where your fingers are supposed to go. Maybe if you hit the wrong key it would flash red. There's all kinds of things you could do.
Ah, so you want this for lessons on a real piano? That seems oddly specific. Far cheaper to get one that has midi capabilities and use any of a number of great apps out there.
I grant that if you can afford a full size piano, you probably don't balk at this thing, though.
https://youtu.be/kLQsUIS01nM