It's a powerful device with superb cameras & sensors, used to play user specific content, running totally unauditable code, connected to the Internet and requiring a real identity account to even start.
If you enable hand tracking (without controllers), they explicitly notify that they are collecting data about your hands. Combine that with your arm length, hand size / shape, height and I bet you'd be pretty unique. If there aren't enough bits, data about "the way you move" or stand, general posture etc. would be more than enough to identify you I believe. A simple DNN can eat that data like breakfast. I don't care much but it is interesting nonetheless.