Is it really that different than a search engine? Take away the AI specific language and you have two products that when given his username return results with his real name.
With classic search engine indexing you can find and remove exact matches from the index, but with neural networks it's harder to make sure you removed every representation of a specific information from the parameters.
For example you remove somehow the exact username-name from the model parameters ( that doesn't seems to hard at first) but then it may still return the information if somebody ask the model differently.
So if you try to remove the information from a neural network model then it can still have it in different forms you may not even think of, for example in language models the same thing described with different words.
And on the other hand removing one thing may affect the models performance on other unrelated things too.
If that's the case, it means that GPT-3 doesn't just raise ethical questions, but legal ones as well: several jurisdictions around the world currently require that search engines allow for the erasure of private information upon request.
Another commenter pointed out that a lot of these models aren't publicly accessible, but will still be used to retrieve information about you - by say employers contracting with a ML company
But they can only be used to retrieve information that is already out there. This is still just using GPT-3 as a search engine, it's just a weird search engine that isn't made to purpose and most of the time produces nice-looking nonsense instead of valid data.
So if you try to remove the information from a neural network model then it can still have it in different forms you may not even think of, for example in language models the same thing described with different words.
And on the other hand removing one thing may affect the models performance on other unrelated things too.