[adverbly]

Reaction time seems to be a lot more complicated than just 200ms flat. There were a lot of superhuman reactions that game which really made it difficult for humans to start a fight. At the same time though, increasing much more than 200ms might give a big advantage to humans in other situations. It will be interesting to see if they can find a better way to model it.

All in all I am very impressed by the AI. So many complex strategies that it is employing(grouping for towers, lane swaps, etc...). But at the same time, it really does look at the moment like this is only close because the AI has vast advantages from mechanics and teamfighting.

[glalonde]

The human would probably be able to blink/cancel if they were expecting it and therefore able to focus on a single thing prior to an event. The bot can focus on everything simultaneously and doesn't really need to expect anything. It gets a signal, it responds within 200ms, no problem. You could program that analytically.

So I would say the superhuman-ness isn't in the number of actions taken, or in the response delay, but in the massive attention bandwidth. I believe they've attempted to even the playing field in the first two, which are easily quantifiable, but I don't know about the latter.

[ehsankia]

To be fair, that's a little like saying Deep Blue had an advantage because it could try out thousands of possible plays simultaneously. That's true, but what makes humans good at Chess for example is that we have a really good "intuition" at which moves are good, and therefore we can prune non-promising branches in the decision tree better.

Similarly here, the AI can definitely do a lot more things at once, but each individual thing they do isn't very smart. For example, they waste money on useless wards or waste time sitting in front of Roshan. We can of course keep pushing the goal post, but I think if the AI can win with the given constraints, it's still a huge accomplishment.

Even more importantly though, it would be interesting to see if the AI is able to come up with new strategies and techniques that weren't known before.

[cthor]

The main situation that this was apparent is Euls on Axe when he blinks onto a target.

Axe's Beserker's Call has a 500ms cast time. Euls is instant. Bots have a 200ms reaction time. Humans have 200-300ms.

The problem is the human doesn't just have to react to Axe blinking on top of them and decide to target him with Euls. Humans also have move their mouse cursor onto the Axe, which for a human is hard to do in 200-300ms (the time they have after reacting to the Axe blink).

A comparable situation that happens a lot is using BKB or Manta to react to a similar initiation. Pros can hit this counterplay much easier because they only have to press a keyboard hotkey, rather than move their mouse to a target first.

[throw_away_777]

One thing to keep in mind is that humans have to process the game from the image on the screen and input through a mouse and keyboard. We have to move the mouse to react to things. The computer is super-human in part because it doesn't have to do these things. It will be interesting to see if they can translate their learnings to bots that react from the image on the screen rather than the API.

[infogulch]

"attention bandwidth" is a great term that really solidifies my fuzzy thoughts on how to characterize the bots, thank you!

[IMTDb]

Can't they "just" add a cost to each API call ? Wanna know the position of an enemy on the map: 50ms. Target a hero/creep/tower: 75ms.

[josefx]

Just make them drop the API and require use of computer vision on the same UI human players have to use.

[conjectures]

Yeah, this is the thing that I wasn't expecting when I first saw this reporting. Preferably with some lag to cursor moves etc.

[akyu]

Saying 200ms flat reaction time is "human level" is really unfair. That is raw reaction time when a is human paying attention for a stimulus, like in a reaction time test.

Lets look at casting a spell in a Dota. First I need to perceive my target. Then I need to move my mouse to the target, which is error prone. Then I need to press a button and click. Even for a human with extremely fast reaction time, this targeting phase will put you above 200ms easily, even if your perceptual reaction time was only 200ms.

The OpenAI bots don't have to target. They process their input, and take an action in just 200ms.

There may be situations where a human can make predictions, pretarget something and get a true 200ms reaction time, but in the general case 200ms is super human by a significant margin.

[JoeAltmaier]

Hm. I used to run an experiment at an Engineering Fair at school. Oscilloscope, random pulse, trigger button, time the reaction time. Some fast kids could get it down to the double-digits, even 20-30 ms, pressing that button after seeing the pulse. If I remember right.

[nkrisc]

That sounds like an interesting experiment. Is it possible they were predicting the pulse? I could imagine a scenario where they would predict and at times get it wrong and press before the pulse, but at times the timing could work out just right such that after accounting for all human latency, the press landed just after the pulse coincidentally. What other factors did your experiment account for?

[citation_please]

Hmm. Using the site that was the first Google result[0] to test myself and I could never get anything below 220ms. Could you be misremembering by a factor of 10?

[0] https://www.humanbenchmark.com/tests/reactiontime/

[akyu]

Those were some exceptional children.

[JoeAltmaier]

Yeah it can't have been so low can it? But it was double digits. Anyway, the fastest ones would put their eye right up to the scope, so the pulse would have the largest signal right into their eye. I thought that was clever.

[akyu]

My guess is there was some tell that they were using to predict the timing.

[def_true_false]

The humans also have the hardware input lag, possibly tens of ms.