[taylodl]

I look at it like this: we don't understand how dogs think, either. We don't know if they're conscious the same way we are. We know they don't see colors the same way we do. We know they can hear sounds that we can't. We know that smell is their primary sense, and they can smell orders of magnitude more than we can. What does all this mean? We don't know. Humans and dogs experience the world differently, yet we get along. We even enjoy companionship with one another. Dogs are similar, but different from us.

I see AI the same way, even non-existent AGIs. Will they be conscious? Probably, but not in a way we'll ever understand. They'll think the same about us. I think the biggest thing we should be working on is mutual respect, ensuring we have a complementary and non-adversarial relationship. Enjoying one another's companionship like we do with dogs.

I suppose what's most worrisome is we understand a dog's selfishness and how their selfish impulses manifest themselves. We collectively have millennia of experience dealing with it and training dogs to not act on those impulses. We simply don't have that experience with AGI. We have no idea how selfish behaviors would manifest themselves, nor do we know what to do if they did.

Finally, it's interesting to note that one thing AGI lacks is direct experience. Essentially, we're working on building hyper-intelligent four-year-olds. Imagine a four-year-old having all the knowledge of humanity, but none of the experience. I think I'm going to stop there - I just terrified myself!