Yes, and some people love it. But the AirPods don't have enough processing power so they need to send the measurements to the host via bluetooth and then the adjusted audio is sent back later. The result is 200+ ms of latency on the head-tracking which is much larger than the perception threshold at 30ms.
I don’t notice any latency issue generally. But, if you change your heads “default position” (i.e. instead of looking forward most of the time you’re now looking out the window next to you most of the time) it does take quite a few seconds to readjust and find the middle.