[JoshCole]

AI research, much like evolution, is strongly in the camp that anthropomorphizing is rational; that human culture often fails to recognize this has more to do with a common intellectual pit that pop psychology and philosophy fall into: when something is clearly in error, it does not follow that it is in error. People often think they can safely critique general methods with specific examples, because the nature of the algorithm that both evolution and AI research condones is to do just that. The thing is, this doesn't reject the algorithm itself, it is what the algorithm does, not a refutation of the algorithm. If you want to actually reject anthromorphizing what you actually need to reject is that in multi-agent decision problems the complexity of the correct solution grows combinatorially with respect to the complexity of the problem such that there are not enough atoms in the universe and not enough time to tractably compute the correct answers such that it makes sense to start with a solution that has error and then improve it in specific situations. As an agent living in that reality, what you see is the constant failure, which you can critique, because it helps you improve, but it is an error to think the tendency itself is in erorr - the error isn't actually irrational, it is more like the speed of light, a physical inescapable law. That is why you see something analogous to anthropomorphizing in the superhuman AI we have made: it shows up in poker AI, in self-driving car AI, in chess AI, in Go AI; actually DeepMind found that if you remove this specific component from the superhuman AI we currently have, they stop being superhuman.

I can link an interesting talk on this subject if you are interested in hearing more.

[joe_the_user]

AI research, much like evolution, is strongly in the camp that anthropomorphizing is rational

Evolution doesn't have opinions so it's not in a camp.

Human behaviors like reciprocity and consideration for feelings are indeed part of human collective behavior. Calling such behavior "rational" misses the point - such behavior exists and we have the benefit of social existence because of it and this bring us benefits collectively. But individual calculating purely individual benefit would naturally just fake social engagement - roughly such individuals are know as sociopaths and they can succeed individually being a detriment to society. Which is to say a social creature is a matter of rationality but simply evolutionary result.

Still, the one thing most people would say is irrational is trusting a sociopath. Now, a Chat bot is absolutely a thing programmed to mimic human social conventions. A view that anthropomorphizes a Chat bot doesn't see that the chat bot isn't going to be actually bounded by human conventions except accidentally or instrumentally, basically the same as trusting sociopath.

[JoshCole]

I am a high decoupler. I generalize things like "analogy to self, self is human" to "analogy to self, self is category X" in order to improve my cognitive abilities by gaining abilities which have reach beyond the confines of what I have previously seen. So when you try to stick with just humans, I'm not with you anymore, because your models seem highly coupled. I find that to be a bad property. I seek to avoid it. I consider it to be incorrect.

In my model, when you talk about anthropomorphism, seemingly as a negative, I realize I've noticed things which a coupled model doesn't predict: that intentional error via anthropomorphism can not just be correct, but that your scare quotes around rational while trying to denigrate the idea that it can be correct could not be more wrong, because the hard to vary causal explanation of why we ought to anthropomorphize gives a causal mechanism for why we ought to which is intimately tied in, not with being irrational, but with being more rational.

I realize this sounds insane, but the math and empirical investigation supports it. Which is why I think it is worth sharing with you. So I'm trying to share a thing that I consider likely to be very surprising to you even to the point of seeming non-sensical.

Would you like a link to an interesting technical talk by a NIPS best paper award winning researcher which delves into this subject and whose works advanced the state of the art in both game theory and natural language applied on strategic problems in the context of chat agents? Or do you not care whether anthropomorphism, when applied when it shouldn't be according to the analogical accuracy that usually decides whether logical analogy can be safely applied might be accurate beyond the level you thought it was?

I am not trying to disagree with you. I'm trying to talk to you about something interesting.