Awhile. This just isn't like Go or Chess. The gap from perfect information to imperfect information is quite a chasm, and from turn-based to real-time is even more vast.
I play Age of Empires 2 semi-competitively, and I just can't imagine the research progress that would have to be made for a pro to lose to an APM-limited AI agent. So much of the game comes down to intuiting what your opponent is planning without being able to see what they're doing, and more importantly intuiting what your opponent isn't ready for.
The biggest difference, though, is the "RT" in "RTS"-- real time. This isn't turn-based anymore, where at a given moment you have a single choice to make, a single piece to move as in Chess and Go, and can then wait for the singular and visible reaction your opponent makes before making your next choice.
My understanding it that the moves a program like AlphaGo makes are not interconnected-- it picks each move individually as an ideal move for that board state. It could take over halfway through the game for someone else and would make the same move that it would have made at that point if it had been in control the whole time and arrived at that board state on its own.
But that doesn't work in a real-time game, since you and your opponent are now moving simultaneously and the "board" is never static. Your moves must be cohesive and planned and flow continuously without time to ponder, each connected to the last. There is no "one" move for a given state.
Another facet of real-time play is the idea of distraction. It's very important in RTS's to keep your opponent distracted, to disrupt their plans and their focus, by coming from unexpected directions at unexpected times, sometimes concurrently with other operations against them. This can't happen in Chess or Go, where the demands on your focus are far less urgent and two things can't happen at once in a literal sense. Can an AI agent learn to appreciate the power of distraction? Can it learn to intuit what will be most disruptive to a human, and what won't be disruptive at all? How can you teach a computer to learn to be annoying?
I will say, of course, that nobody saw AlphaGo coming. And I hope it's the same with RTS's. That would be so exciting. I would love to see an AI blow us away with previously unthought-of strategies. That would be the coolest thing ever. So I hope it happens. But I'd be astonished. RTS is just such a whole new level of thinking for AIs.
At pro level, much of the game is about what information you can gain, and about choosing what to show and, more importantly, what you don't show (hide) and acting on non-triggers.
An example of a non-trigger is knowing that if I haven't seen a certain unit at time X, I know I'm safe to do Y. It is acting upon the information that something didn't happen.
To expand: I saw my opponent starting two gases at my 21 supply scout. When I scouted again at 47 supply, I saw no gas heavy units, so I can deduce the gas was used for better technology. This will allow me the opportunity to increase my worker count by Z before building army, or I could try and kill my opponent right there for his technological greed.
I haven't played AOE2 so I don't know if the mechanics are similar enough to translate, but my goal for my Starcraft bot is to do precisely this. If you can enumerate the possible builds (what's available when) and assess the matchups between builds, you can make this happen using some intuitive expansions on adversarial search.
> Your moves must be cohesive and planned and flow continuously without time to ponder, each connected to the last.
Recomputing the entire plan from the current state works in RTS too, but only if your decision-making takes every already-in-motion thing into account and has no internal discrepancies. That's a pretty big if; this sort of weakness accounts for a lot of bot weakness currently. Units spinning around due to slight changes in perceived state cause lots of wasted resources.
> Can an AI agent learn to appreciate the power of distraction?
Despite multitasking theoretically being one of the strengths of an AI, a lot of the current field can't handle more than one military situation at a time. In this year's SSCAIT a lot of bots completely fell apart when confronted with one of the top bots (Bereaver) doing reaver drops.
I'm not sure a bot can meaningfully learn distraction, but I'm not sure it's necessary - attacking on simultaneous fronts is optimal anyway. The army can only be so many places at once.
On the other hand, bots are starting to beat professionals at (thousands of hands of repeated) Poker, so I think we can't say that imperfect information is something that's especially intractable for maching learning algorithms.
Exactly. And poker is, again, turn-based with a single move to be made. And it may not be "perfect information," but it again is a game of static board state where there is probably a statistically optimal move for a given state provided you have memory of the other players' previous moves so far in a round.
Speaking of AoE 2, I really hope this research ends up benefiting that game. The AI in it has always been so bad but it's still my favorite game of all time.
Microsoft hired one of the best scripters (Promi) on the AoE2 AI circuit to write the official AI for AoE2 HD, and it's quite good, although the rules engine doesn't have enough features to beat early harassment. If you let it boom it's pretty scary for new players.
If you play using the UserPatch on Voobly, where the serious custom AIs are written, you can play vs. Barbarian. It is _very_ good, if you're not a semi-competitive player it will certainly beat you.
The upcoming UserPatch 1.5 will add even more features, so Barbarian and other custom AIs will become stronger.
To be clear, AoE2 AIs are all rules-based and written by pro players themselves, which is quite different from what DeepMind is trying to do.
It is hard but my impression is it's because it cheats. I'm more looking for interesting AI (creativity, adaptation) rather than raw difficulty due to its ability to spam military.
Ooh, this'll be interesting to see, as with AlphaGo, a lot of people disputed the "experts believed it would take ~10 more years" claim retrospectively.
With SC2, no AI even comes close to beating even a silver level player, so even a 5 year timeline seems really soon. Let's see if DeepMind can beat it!
I think it is doable in under 5 years, but this critically depends on the resources invested by DM and other DL orgs. Deep RL is hugely demanding of computational resources to iterate your designs - for example, the first AlphaGo took something like 3 GPU-years to train it once (2 or 3 months parallelized); however, with much more iteration, DM was able to get Master's from-scratch training down to under 1 month. Now an AG researcher can iterate rapidly with small-scale hobbyist or researcher resources, but if they had had to do it all themselves, Ke Jie would still be waiting for a worthy adversary... When I look at all the recent deep RL research ( https://www.reddit.com/r/reinforcementlearning/ ) I definitely feel that we can't be far from an architecture which could solve SC2, but I don't know if anyone is going to invest the team+GPUs to do it within that timeframe. (It might not even be as complex as people think: some well-tuned mix of imitation learning on those 500k+ human games, self-play, residual RNNs for memory/POMDP-solving, and use of recent work on planning over high-level environment modeling\, might well be enough.)
I can't decide if I would be bummed or excited if that turns out to be the case. On the one hand, we'd be that much closer to AGI. On the other, we'd be continuing down the path of brute-forcing intelligence, rather than depending on those elegant, serendipitous breakthroughs that much of human progress has been built on.
Do you have a citation? As far as I know, existing AIs do not stand much of a chance again humans [1]. My understanding is that even if AIs can beat humans is small skirmishes due to superior micro, they lose so badly at macro that it simply doesn't matter.
I don't want to be overly semantic or PC on HN, but just saying the GP may be female, judging by their name on profile. Being misgendered could be very offputting and discourages participation, so you may have wanted to say "this person" even if it doesn't sound as offhand as you'd have liked it to come across.
Calling a group of women "you guys" is different from calling a person of unknown gender "guy" because in the former case the gender-neutral usage is the only possible interpretation, whereas in the latter it might either be gender-neutral usage or an assumption of the commenter's gender. Given that this ambiguity is nearly always a distraction from the primary purpose of the conversation (as it is here), rightfully or not, pragmatically it is better to use terms that steer clear of it. See Bryan Garner's Modern English Usage for fuller advice on safely navigating this linguistic and sociocultural terrain.
it's not exactly macro.. AI can run perfect macro commands. humans right now have better priority model (in this case, protecting/killing workers) for which the AI in question wasn't trained.
Are you sure you're interpreting 'macro' correctly? 'Macro' in the context of RTS games isn't the same as 'macro commands' in scripting, but refers to basically all high level decisions. RTS is divided up into Macro, tactics and micro:
- Macro: resource prioritization (army vs expansions vs upgrades), scouting to understand the opponent's macro choices, choosing the right posture in response (defensive, harassment or offensive), and resource optimization (not getting supply blocked, scaling production with income, increasing income at the maximum possible rate, removing bottlenecks, etc), scouting the enemy army composition to prepare the ideal counter army composition
- Tactics: Grand army decisions - flanks, baits, sneak attacks, hiding composition, multiprong attacks, positioning of siege units, timing attacks, knowing when to retreat (hit & run), scouting to gain advance notice of your opponent's tactics
- Micro: optimizing unit lifespan and effectiveness within an isolated skirmish for the given goal (usually to 'win' the engagement) -- pulling back weakened units to avoid aggro while it still deals damage, healing, surface area for melee units, trapping enemy units with terrain or skills, optimizing spellcaster energy usage, prioritizing targets based on multiple parameters (range, damage, cost, count, follow-ups), etc.
AI can "run" macro well, but they are poor at the macro decision-making part, which includes priority model as you mention (the responsive posture choice above and others). Up to low grandmaster tier, being significantly better at macro than your opponent while close in tactics and micro is usually enough to win consistently. It is the most impactful part of an RTS (and is where most of the 'S' lies).
Your definition of macro is a lot more encompassing than how I see it used in SC2. It normally refers to just spending resources properly (constantly producing units and workers, setting up infrastructure, etc).
Yeah, interesting! I agree I don't usually hear macro used that way.
But if someone was microing a battle and as a result didn't look at the minimap and see a drop arriving at their base, I can totally see concluding that this player is bad at macro as a result -- macro is referring to there being a macro cycle of tasks you have to perform all the time whether you want to or not, and non-production tasks like checking the minimap and sending in a scout seem like good examples of those tasks too.
I hear you. Usually the bottleneck of macro-effectiveness is the part you mention, the decision-making parts can be simplistic in mid-to-high levels and still plenty effective. So "improving your macro" usually refers to remembering to build workers and not getting supply blocked. Getting supply blocked or not hitting production cycles is more likely to get you behind in StarCraft.
That said, in high-level play, "better macro" usually refers to the other things, not just mechanically hitting stride with production, as most players in the top .01% are on the same level with those mechanics.
There isn't any SC2 AI that's anywhere near as good as a player in an actual game. Sure, they can demonstrate some sick unit control, but that alone isn't enough to win games.
I play Age of Empires 2 semi-competitively, and I just can't imagine the research progress that would have to be made for a pro to lose to an APM-limited AI agent. So much of the game comes down to intuiting what your opponent is planning without being able to see what they're doing, and more importantly intuiting what your opponent isn't ready for.
The biggest difference, though, is the "RT" in "RTS"-- real time. This isn't turn-based anymore, where at a given moment you have a single choice to make, a single piece to move as in Chess and Go, and can then wait for the singular and visible reaction your opponent makes before making your next choice.
My understanding it that the moves a program like AlphaGo makes are not interconnected-- it picks each move individually as an ideal move for that board state. It could take over halfway through the game for someone else and would make the same move that it would have made at that point if it had been in control the whole time and arrived at that board state on its own.
But that doesn't work in a real-time game, since you and your opponent are now moving simultaneously and the "board" is never static. Your moves must be cohesive and planned and flow continuously without time to ponder, each connected to the last. There is no "one" move for a given state.
Another facet of real-time play is the idea of distraction. It's very important in RTS's to keep your opponent distracted, to disrupt their plans and their focus, by coming from unexpected directions at unexpected times, sometimes concurrently with other operations against them. This can't happen in Chess or Go, where the demands on your focus are far less urgent and two things can't happen at once in a literal sense. Can an AI agent learn to appreciate the power of distraction? Can it learn to intuit what will be most disruptive to a human, and what won't be disruptive at all? How can you teach a computer to learn to be annoying?
I will say, of course, that nobody saw AlphaGo coming. And I hope it's the same with RTS's. That would be so exciting. I would love to see an AI blow us away with previously unthought-of strategies. That would be the coolest thing ever. So I hope it happens. But I'd be astonished. RTS is just such a whole new level of thinking for AIs.