[wavyknife]

(disclaimer: I work for Discourse)

Discourse has an AI plugin that admins can run on their community to generate their own sentiment analysis (among other things), though it's not quite as thorough as this write up! https://meta.discourse.org/t/discourse-ai-plugin/259214

We're always interested to see how public data can be used like this. It's something that can be a lot more difficult on closed platforms.

[Aachen]

> helps you keep tabs on your community by analyzing posts and providing sentiment and emotional scores to give you an overall sense of your community for any period of time [...]

> Toxicity can scan both new posts and chat messages and classify them on a toxicity score across a variety of labels

Is that within the defined data processing purposes of all Discourse setups? Does the tool warn admins they might need to update their policies before being able to run this tool, perhaps needing to seek consent (depending on their jurisdiction and ethics)? It sounds somewhat objectionable, trying to guess my mental state from what I write without opt-in

Edit: and apparently it also tries to flag NSFW chat messages, does Discourse have PM chats where this would flag private messages for admins to read or is it only public chats that this bot runs on?

> tagging NSFW image content in posts and chat messages

[eddd-ddde]

I don't think there's anything left for you to consent once you decide to post on a public forum. If I can read your post and guess your mental state so can any other bot.

[Aachen]

If you park your car on the side of the road, that also doesn't allow anyone to do with it what they please

If you write an article and post it on your blog, people can't just come along and take the text verbatim

If you license your blog as public domain, then someone takes the content and does something objectionable with it, you can (in many countries) still make use of moral rights if you'd wish to correct the situation

If I post something publicly on a forum, I'm well aware I may have agreed or consented (depending on the forum) to terms that allow this type of processing, but that is not the default. There exist restrictions, both legally and morally (some legal ones are even called moral rights and are inalienable). Hence my question how this plugin handles extending the allowed data processing to cover taking the content and making automated decisions and claims that may or may not be accurate. I would not be comfortable with that being an automated behind-the-scenes process flagging my posts as good or bad towards the moderators, since they likely won't care to read back hundreds of comments and see whether the computer did a good job

[wavyknife]

Discourse is not a centralized platform, so it's up to individual sites to ensure they're compliant with data and privacy regulations.

[Aachen]

I mention that in the first nonquote sentence

[BadHumans]

More companies and communities than you think already do this without your knowledge let alone consent.

[david_allison]

That doesn't mean we can't do better

[BadHumans]

Better at what though? I don't even think it's a problem to begin with.

[xfalcox]

> Is that within the defined data processing purposes of all Discourse setups?

It's an optional plugin that can be enabled / disabled by the site admin. Those modules are all disabled by default, and each need to be enabled by the site owner.

> Edit: and apparently it also tries to flag NSFW chat messages, does Discourse have PM chats where this would flag private messages for admins to read or is it only public chats that this bot runs on?

Discourse PMs can be read by admins, see the definition here: https://meta.discourse.org/t/guidance-and-best-practices-on-...

[Aachen]

Of course an admin can always open up the database and read your forum PMs, that's not surprising. The very first line in the link you provided, however, is what I was worried about:

> Moderators can read PMs that have an active flag.

This system is now setting nsfw flags in an automated fashion, specifically seeking out content that the persons involved wouldn't want others to see. Clearly a forum is the wrong place for that content, but people don't always make good decisions (especially kids; I was a kid on forums too and would be very surprised if nothing ever transpired there). The receiving person can already flag anything they deem inappropriate. A system making automated decisions about messages that were intended to be private creates problems and it is not clear to me who this serves

[wavyknife]

> it is not clear to me who this serves

customers