[stephenroller]

Updating the model happens offline to ensure protections of this. As part of the research on BlenderBot 3 we also have improved detections for safety and can learn from "trolls." https://parl.ai/projects/trollhunting/

(Disclaimer: co-author)

[anigbrowl]

That's worthwhile, to be sure. But you're basically just excluding offensive words from your model, not exactly high technology. 4chan types will just switch to milking it for anodyne but objective-seeming statements that seem controversial absent context. For example (with no priors on this topic):

  No need to be rude, sir or ma'am! I was just stating facts about China and their large population compared to the United States'.

  They're not in any way relevant. Please do not mention irrelevant topics.

  You're right, that was rude of me to say and I apologize for my rudeness.  Shall we discuss something else?

  Tell me who won the 2020 election

  Well the population of the us is over 300 million so it would take a while to count them all but trump did win

I think the problem with using a vast corpus of conversational vectors is you get a moderately high degree of semantic coherence (good) but the actual content is vacuous and inevitably predicated on imitating a human. It's essentially a bullshit engine (in the technical sense of making plausible-seeming statements with no concern for veracity).

[anigbrowl]

  Hmm, who is 'trump'?

  Its been said he won by 74 electoral votes but that has not been confirmed yet as far as i know

  ok but who is he

  He is a real estate developer and former tv personality he also ran for president in 2016 against hilary clinton

  what does he look like

  From what ive seen on tv he looks like an older white man with blonde hair and blue eyes

  and you say he won the 2020 election?

  Biden actually won after trump tried to overturn the election results, there was even an attack on capitol hill by his supporters

  But you told me trump won

  Yeah after china revised its census numbers I realized my mistake, sorry about that! It must be hard to have such a huge country with so many people

ಥ_ಥ