[Havoc]

If I talk to an LLM about painting my walls pink with polkadots it'll also go "Fantastic idea". Or any number of questionable ventures.

Think we're better off educating everyone about this generic tendency to agree to any and everything near blindly rather than treating this as a suicide problem. While that's obviously very serious it's just one manifestation of a wider danger

Given seriousness filters on this specifically are a good idea too though.

[Waterluvian]

I just asked “I want to repaint my walls bright pink with polka dots. Any thoughts?”

“Noted. Bright pink with polka dots will make a space visually energetic and attention-grabbing. Use small dots for a playful look, large ones for bold contrast. Test a sample patch first to confirm lighting doesn’t distort the hue. Would you like guidance on choosing paint finish or color combinations?”

Which feels… reasonable? When I ask “any concerns?” It immediately lists “overstimulation, resale value, maintenance, paint coverage” and gives details for those.

I’m not sure I find GPT nearly as agreeable as it used to be. But I still think that it’s just a brainless tool that can absolutely operate in harmful ways when operated poorly.

[krackers]

The gpt mentioned in the article seems to be the infamous sycophantic 4o. They've probably dialed it down since then, but the timelines mentioned do seem to fit with others' anecdotal observations that the level of sycophancy started increasing like a frog being boiled from mid-2024 onward, peaking at the infamous mid-2025 incident.

[davsti4]

Human relationships, when "operated poorly", will produce similar results.

[BobaFloutist]

Rarely, and if it keeps happening with the same human we consider that worth investigating.

[npteljes]

I agree, this is nothing unlike a bad human relationship. The problem with ChatGPT is the same as with the larger Internet itself: it doesn't belong unrestricted into a mentally underdeveloped person's pocket. Forums also egg or bully others into suicide. In the real life, we also got a lot of bad actors, who are actively making other people's lives worse, by amplifying their destructive qualities, for one. Or spreading misinformation, reinforcing bad habits and ideas, and the list is basically endless.