Hacker News new | ask | show | jobs
by DuskStar 2574 days ago
This feels a lot like OpenAI 5, but with a few key differences - for one, it seems to handle randomly generated maps and other unexpected situations, and each agent does not have full knowledge of the others on its team. Really cool!

And it would be hilarious if this ends up giving us video game enemies that don't suck. (Until, of course, someone accidentally creates an RTS AI that manages to 'break out' and conquer the world or something)
2 comments
stefs 2574 days ago
evolving agents like those are very good at finding bugs to abuse. so, if they'd find a zero day to break out of the quake-sandbox, what would they do? i guess, for the longest time they'd just use the DMA into the sandbox to increase their score (i.e. cheating by developing and using external tools), as their fitness function still weights CTF wins only.

to (accidentally) change their fitness function to change it to "conquer the world" they'd need to modify a lot of different processes at once.

IMO the chance is negligible for now.

link
DuskStar 2574 days ago
I think the (very sci-fy) fear there isn't "we gave this agent a goal of 'score xyz points' and it escaped the sandbox to increment the points counter" but instead "we gave this agent a goal of 'conquer the world' and made it think that the game it was in was the world, and then it discovered otherwise"
link
pygy_ 2574 days ago
I wonder who from Google, Facebook and OpenAI will be the first to productize a "CEO as a service" AI.
link
wnkrshm 2574 days ago
I think it's an interesting question when this will start to show up - not in terms of managing everything including taxes but in terms of making automated business decisions.

In niches where business decisions are data driven, connecting a supply in some market to a demand in another, the choice of what to buy and sell could conceivably be automated.

Like gauging demand for mug/thermos/branded item du jour on amazon, checking whether similar items exist on alibaba and then setting up an amazon market place that resells your slightly rebranded but identical item from alibaba. Of course this all hinges on the data - an algorithm is prone to fall into traps that some humans fall into (i.e. the company you buy from only sells the box and not the product).

link