[chongli]

Moreover, these were designed by humans with very explicit successes, losses and easily distinguishable outcomes.

This is only true if you assume a TON of contextual knowledge of biology, human culture, civilization, warfare, etc. Why should an AI have any of that?

Look even at an extremely simple game such as tick-tack-toe (or noughts and crosses). How should an AI learn to play this game? Assume it has been given no knowledge of the game beforehand, like a human child shown the game for the first time. How should the AI know who wins and who loses the game? How should it know that the objective is to win at all? The idea that winning is desirable seems to be hard-wired into human beings; why should that be the case for an AI?