Can you prove that any other person you know is generally intelligent, and not just a LLM spewing statistically relevant words ?
You assume they are intelligent based on their actions and abilities, so I guess we will do the same with AI.
Fake in this context is not any more dangerous than a con man (wants to harm, but it’s not competent), which is already pretty dangerous, but not as dangerous as somebody who is capable and interested in harming.