[grues-dinner]

It would be a fascinating thing to happen though. It makes me think of the Greg Egan story Unstable Orbits in the Space of Lies. But instead of being attracted into religions based on physical position relative to a strange attractor, you're sucked in based on your location in the phase space of an AI's (for whatever definition of AI we're using today) collection of contexts.

It's also a little bit worrying because the information here isn't mysterious or ineffable, it's neatly filed in a database somewhere and there's an organisation that can see it and use it. Cambridge Analytica and the social fallout of realtime sentiment analysis correlation to actions taken has got us from 2016 to here. This data has potential to be a lot richer, and permit not only very detailed individual and ensemble inferences of mental states, opinions, etc., but also very personalised "push updates" in the other direction. It's going to be quite interesting.

[kaycey2022]

I wouldn't call it fascinating. It's either sloppy engineering or failure to explain the product. Not leaking user details to other users should be a given.

[grues-dinner]

It would absolutely be fascinating. Unethical in general and outright illegal in countries that enforce data protection laws, certainly. Starting hundreds of microreligions that evolve in real time and bring able to track it per-individual and with second-by-second timings, and being able to A-B test modifications (or Α-Ω test, if you like!) would be the most interesting thing to happen in cognitive science ever and in theology in at least centuries.