[harmmonica]

This seems like a good argument for some kind of AI guardrails. And then the question is who decides where the guardrails start and end? I guess the straw man is: if you knew the AI friend was not going to advocate destructive behavior, to oneself (maybe less important?) and to the world around you, then the AI friend might actually be a net benefit because the actual person may feel more emotionally supported than they otherwise would be.

The problem is there's almost no way to regulate the guardrails, at least in a country like the US, and so then you're left with corporate interests deciding on the guardrails with their only incentive being hockey stick growth in engagement/revenue, which we already know is destructive in almost every corporate vertical (social media, food, etc.).