[JamilD]

The AI can minimize loss / maximize fitness by either moving to look for additional resources, or fire a laser.

Turns out that when resources are scarce, the optimal move is to knock the opponent away. I think this tells us more about the problem space than the AI itself; it's just optimizing for the specific problem.

[swalsh]

I think you're right, had the AI (like the second game) had the incentive to maximize the well being of the cooperating actors, rather than itself, the outcome would be different.

But if advanced AI is being developed in a capitalist economy by independent actors, it seems most likely the incentives will be anything other than optimizing the output for the individual outcome.

If that AI finds a way to "hurt" the other actor, there could be some major boat load of unintended consequences.

[laretluval]

> I think this tells us more about the problem space than the AI itself; it's just optimizing for the specific problem.

My reading is that the point of this research is to find out what problem spaces are conducive to cooperation, not to find out details of how these particular agents work.

[JamilD]

Yeah — after reading this article and the paper, I agree. The previous link [0] on this submission was much more sensationalized and very misleading.

[0] http://www.sciencealert.com/google-s-new-ai-has-learned-to-b...