[daveguy]

Superhuman has a very specific definition in the field of game playing algorithms. It is the case when the algorithm can always beat all humans. AlphaGo winning 5-0 against the number 5 ranked human (Lee) would give only a small indication that it is superhuman. Regarding streaks, evenly matched humans can go 5-0 an expected (0.5)^5 or 3.125 percent of the time. So not particularly rare. If it loses even one game then it is not yet superhuman.

If top humans get beat 5-0 with significant handicaps then it is likely AlphaGo is superhuman. However, it is expensive to run AlphaGo so it is unlikely that we will know the true strength of AlphaGo for a while (more challenges) or until hardware catches up.

Update: typos and clarifications

[YeGoblynQueenne]

The term ("superhuman") is used very differently in other communities though, that don't have such a clear metric of "beating humans" as in traditional game-playing. Which means it's about time we clarify what is meant by it, especially as various parties that have commercial interests start throwing it around carelessly (there was an example on HN a while ago).