I am pretty sure telemetry is sent asynchronously and shouldn't affect editor's performance. Also, if you have your doubts, you can easily opt out. It's very visible in the Settings.
It's interesting they see a noticeable difference. I don't think it's the calls themselves that cause the latency, probably all the code trying to figure out every single thing you're doing within VS Code to send over.