[rnosov]

How different is this demo from just having input text box with suggestion to enter your name in it? If a user is foolish enough to divulge their name to pirate accented bot perhaps you might bypass it and ask for it directly.

[greshake]

Because users obviously trust Bing's output not to be directly controlled by an attacker. The pirate accent is optional. Bing can also exfiltrate any other information in any other Tab that it sees or that users enter. The injection can also happen on social media, like in a Twitter thread. Users can even navigate to other tabs and the injection will remain active (at least for some time). "Clearing" the conversation with the broom doesn't help either.