I think he's running into the issue that when the phone is playing loud audio through the speakers, it has trouble picking up voice over that. iPhones don't have the same audio separation hardware as HomePods.
Yes, this is why my initial comment was a bit snarky. GP wants his phone to have properties it fundamentally cannot have, and is not expecting that of any other phone.