[outworlder]

> We learn our values, just as an AGI would

Yes, but there's a problem. An AGI would be a completely alien mind. It does not have to have the same values(or any values we could relate to), just like you wouldn't expect a super intelligent spider to have any morality, no matter how good the parenting was.

[wapapaloobop]

An AGI would have to begin with the same values because without them he couldn't learn anything. Our values draw on the surrounding culture and an individual that grows up outside of any culture (e.g. a feral child) isn't capable of functioning. There may be different cultures but there is no alien culture. Thus there could be no alien mind.