[tialaramex]

This specific attack doesn't impact your usage scenario. It is impossible to say with certainty whether a hypothetical attacker, who had stolen one of the NEOs enrolled in your system, and had suitable lab equipment, could conduct a similar attack to recover authentication credentials from the NEO if they stolen the PIV PIN. Perhaps, perhaps not.

In general you should not be worried about this, it is unlikely you are so well defended that "Buy this lab equipment, hire an expert, and then steal someone's Yubikey" is the most viable attack, so time spent figuring where the low hanging fruit is will be better than worrying about this.