[neumann]

Approach it from the other angle - what scenario would it be bad in. It's not hard to see very real possibilities in the short term where it does matter: A 16yro looks up on chatGPT how she can check discreetly whether she is pregnant or not and what potential avenues she has. The advertiser could literally by anyone targeting pregnancies, including government or action groups who now have some information about that user's conversations in this scenario.

Any data exfiltration or reporting on the users would quickly be developed by the industry to merge this information and improve inferences with confidence values on target populations/individuals.

[charcircuit]

You don't need ChatGPT to know that someone buying a pregnancy kit from you has an above average chance to be pregnant.

[wizzwizz4]

Who said they had to buy it to leak the information?

[charcircuit]

Replace the word buy with view and it will still be true.

[wizzwizz4]

But if they're only viewing it because it was put on their screen by ChatGPT, that's a non-consensual leak of personal information to the advertiser.