[hacknat]

I believe AGI is possible, but the more I study ML the more convinced I am that we don’t know what it (AGI) is. I also, don’t know if I agree that humans have GI (humanity as a whole does). I think we’re actually copycats more than anything else. Neural nets are the atoms of GI, but it’s not clear what the superstructure of GI is. We all assume that we understand what general intelligence is, but I don’t think we actually have the goods yet. We understand optimization, but we don’t understand the thing that chooses what to optimize. There is some notion of “values” in order for intelligence to be legible, and we don’t know what that is either.