That should be the case, but the frequent data breaches that I get notified about with little consequence to the companies involved make me doubt that there will be real incentives for good behavior.
On doctors, I read a substack by a doctor who stopped using an AI notetaker after trying it for a year. They weren't his notes, and there was too much extraneous detail.
Can't find it with a quick search, but the point of the essay was that making the notes by hand reinforced the essentials of each case in the doctor's mind, so they were there for the next appointment two weeks later.
I have usually permitted that when it comes up and sometimes declined it. But when I permitted it I didn't have to worry about them entering everything I say into a permanent history from which a random sentence could get used, datamined, hallucinated into something that could be used against me years later.
Medical students are covered by rafts of privacy laws that will end their careers if violated.
The modern tech industry runs on privacy violation.
The two are not comparable.