[layer8]

> you're talking to distillation of humanity, as a whole, in a box.

This is an aside, but my impression is that it is a very selective and skewed distillation, heavily colored by English-language internet discourse and other lopsided properties of its training material, and by whoever RLHF’d it. Relatively far away from being representative of the whole of humanity.

[jerf]

Yes, absolutely. I'm not trying to claim it's some sort of unbiased sample, but more get across the idea that modelling AI as a person, a singular person, in your head is inaccurate. That singular person would have a stereotypical, Hollywood-esque multiple personality disorder like no actual human on Earth has ever had. You need to be thinking about not just what the person-like thing in front of you is doing, but how to craft which person you're ending up with.