[thallium205]

Pianist hands and fingering don’t really look like that while playing.

[londons_explore]

I think it's because the 38 muscles they have given their robot hand don't map perfectly with real muscles in a real hand.

In particular, there seems to be not much ability to move each finger left or right. For example, actuator 14 lets the baby finger swing outwards - but now try to stretch your baby finger out, and you see it can swing much further than this robot finger can swing out, and the pivot is closer to your wrist, allowing more reach.

[londons_explore]

I suspect these simulated keys have no weight/spring back force. Ie. the robot hand doesn't have to 'push hard' to make them go down.

It would be equivalent to you trying to play a holographic piano in the air in front of you. I suspect you'd adopt a very different hand position too.

[kzakka]

We're using the MuJoCo simulator under the hood: https://mujoco.org/

The keys are actually implemented using a spring mechanism, but springs in MuJoCo are currently linear, which isn't the case in the real world.

[sfblah]

I can play the songs on the list, and the fingering is completely wrong.

[kzakka]

The fingering we use is taken directly from: https://beam.kisarazu.ac.jp/~saito/research/PianoFingeringDa...

From their whitepaper: "Fingerings in the dataset were provided by experienced pianists who graduated from a music college or who had played the piano for more than twenty years. The pianists were asked to choose pieces that they could play and provided the fingering that they had actually used for the performance."

[sfblah]

Very interesting. The fingering for the Chopin piece is just bizarre to me, but I'm not going to argue with data.

[ubj]

Fingering isn't a fixed "right" or "wrong" matter. Every pianist's hands are different, so fingerings may vary from person to person.