[dependsontheq]

I have been working on a monitoring and prebunking system for digital manipulation and desinformation. We are focusing not on the content or narrative but on the psychological patterns and manipulation techniques that are used.

It's the most disturbing thing I have ever worked on, there is much more out there than moste people realize and a lot of it uses deceptive dark patterns.

If somebody is interested in talking more about this or is working on similar things, always welcome!

[tardedmeme]

How do you convince people that your system is not itself disinformation?

[dependsontheq]

I get that question a lot and I can understand it, so there are two things here at work, one is freedom of speech. Depending on your local rules (I am from Germany so our rules i.e. concerning the Nazi past are different than the US rules) freedom of speech should be guaranteed, and opinions shouldn't be labeled as desinformation or manipulation based on their content.

What we are monitoring are deceptive patterns on a text or transcript level. Deceptive patterns can be things like information inconsistency in one post, context shifts in one post that are used to reframe something, or video patterns like fake statistics or fake headlines that are not consistent with the main content.

All of these patterns are actual science backed psychological manipulation patterns and they are consistently used in the most viral posts we detect. My perspective after one year working on this is that the average media literacy is even lower than we think and that we build an evolutionary system with the social media platforms that is optimized to increase the performance of digital manipulation actors.

[Mezzie]

This is the area I wanted to work in before I got sick and ended up being pretty worthless.

I have the following questions, in no particular order - I'm writing this comment off the top of my head in stream of consciousness while procrastinating:

- Why did you decide on a new technical system/platform as the way to go? This might sound like a silly question, but one thing I've noticed when talking to techies who are interested in this problem is that it can veer into 'I have a hammer, look at all these nails!' The reason I ask this is one thing I've noticed in working with the average person (or even very educated non-techy sorts) is that they view themselves as having little to no agency when interacting with technical systems, and I worry that adding another one just further encourages them to outsource that agency (just believing you instead of randos on Instagram/TT). This is versus things like outreach through different channels, formalized educational programs, producing of children's educational material (teach the parents while they read to their kids, for example, which lets them set aside the ego of being lectured to as an adult), traditional/alternative media campaigns, etc. If it's just a case of that's your skill set and resources, fair enough!

- You mentioned in a sister comment the low rates of media literacy. I 100% agree. Do you have/have you found any good ways to handle the combination of high education/socio-economic class and low media literacy? I've noticed very similar patterns across education levels, but my peers with graduate degrees or some manner of social 'success' fully believe themselves to be media literate and in fact some of them could recite most of the deceptive tricks and point them out if asked. They still knee-jerk believe things that confirm their priors.

- Is there any educational focus on heuristics and ways that the average person can satisfice their way to something better than the status quo? A lot of effort in this area seems to assume some platonic ideal of an informed, rational citizen with plenty of time to dedicate to educating themselves/learning better habits. Because of this, they tend to be information dumps. In addition to the low media literacy, there are a lot of people (at least here in America - I can't say for your education system) who lack the requisite knowledge to understand what you're telling them. I know that could sound a little insane, but a lot of people can't manage hypotheticals or understand second-order effects. We've also got the studies about people's attention spans. Going through what amounts to paragraphs of psychological text (or video) presented in a factual way will make people scroll or their eyes glaze over, but actions they can take in their life (e.g. stopping social media use for a month and noticing how their thoughts change, specifically following a small group/topic that you don't belong to/have much interest in to see how conversations change over time without personal investment, etc.) might be more approachable. Right now, the two approaches seem to be 'let experts educate you so you can learn a byzantine system in your 45 minutes a day of free time' and 'just go touch grass/let's go back to 1990'. I don't think either of those are realistic for the average person.