[GuB-42]

Projects like DeepBlue and AlphaGo are not fundamental innovation nor research, they are just PR stunts that show the expertise of the company making them.

TBH, winning a game of chess or go has little value in itself, except for the limited market of selling chess or go software. The reason they are doing that is mostly for publicity. IBM makes computers, and they show how good they are at it by having one beat top players at chess, and Google makes machine learning based products and they use AlphaGo to show how good they are at it.

Chess and go don't drive innovation, they are just a side effect of real innovation.

[dsacco]

While I understand your cynicism in the practical applicability of a chess or go-playing AI, I think you are significantly underestimating the theoretical innovations contributed to the field every time these models are substantially improved. Much of the work that goes into improving something like AlphaGo is cross-applicable and cross-pollinated to other research projects, and gradually trickles out into other domains with much more real-world impact.

[eli_gottlieb]

The basic problem with AlphaGo Zero is that the state of a Go game is fully deterministic, fully Markovian, and fully amenable to quick simulation. The player makes a move, and the simulator computes the next game-state in milliseconds from only the current game-state. This is what lets the AlphaGo Zero agent train so quickly on self-play.

If you start requiring high-dimensional empirical data where the generating dynamics aren't Markovian (or aren't neatly predictable with a Markovian simulator, even if God considers them fully determined), you start having to do stuff like full-blown physics simulations while also specifying agent goals in terms of those physical states. Then you've got the machine learning part and the simulation part taking up comparable amounts of compute power, and self-supervised training becomes much more difficult.

[dsacco]

I agree that partial observation and imperfect information present computational difficulties to generalization. Do you know of any interesting research offhand for reading about optimizations for this problem?

[JustFinishedBSG]

> I think you are significantly underestimating the theoretical innovations contributed to the field every time these models are substantially improved.

I think you are overestimating, there isn't a single interesting theoretical insight in AlphaGo's papers.

[dsacco]

Can you define what you mean by “theoretical insight”? It’s true that AlphaGo was built using previously existing techniques (supervised learning, large dataset for training, reinforcement learning and monte carlo tree search). But if you consider something to not be a breakthrough because it does not literally introduce a novel fundamental technique, you have a very narrow view of research (in my opinion).

Here are a few points to consider:

1. The combination of the aforementioned techniques in AlphaGo was non-standard. Reinforcement learning bootstrapped supervised learning, before passing a value function to the monte carlo tree search.

2. AlphaGo represents a new achievement in solving perfect information games. The research team has moved on to Starcraft, which is not perfect information, but they didn’t try to tackle that before conquering a complex perfect knowledge game first.

3. AlphaGo’s research team improved upon the original AlphaGo with a novel algorithm for self-learning and mastering games using minimal policy improvement. The new AlphaGo Zero does not utilize human training data or supervised learning, and it was capable of defeating the original AlphaGo 100-0.

Beyond self-play, I think that AlphaGo’s methodologies can generalize to combinatorial search problems even if they don’t generalize to broader domains like partially observed games or robotics.

[pinouchon]

I think folding the update rule inside the MCTS loop (in alphaGO Zero) is genius.

[dwaltrip]

That is a big claim. Where's the detailed analysis and citations?

[ionforce]

Playing devil's advocate. I casually agree that AlphaGo Zero was valuable, but if we were to put the onus on you...

What theoretical innovations did AlphaGo Zero provide?

[dsacco]

I gave a brief overview of that in a parallel comment on this thread :)

[mda]

Funny that up until 2016 go was regarded as one of the most difficult games that computers could master, and now that it is solved it becomes a PR stunt? Would you claim the same in 2015?

[GuB-42]

It was a hard problem, and that's what makes it an effective PR stunt.

Google was working on machine learning for some practical application like image classification, better targeted ads or whatever thing Google does. A bunch of people then came up with the idea: "hey, we have all that AI stuff, we may be able to use it for computer go". And Google replied with "OK, sounds like good publicity, here is a budget, we also have a bunch of servers and if you need help, feel free to ask our machine learning department".

It is like making an industrial robot that can crush concrete blocks or whatever difficult but not that useful task. Maybe it is a huge deal because all previous attempts failed, but the point here is not that years of research in concrete crushing robots have payed out, but rather that recent advancement in practical engineering made it possible, and maybe even easy.

[fjsolwmv]

That's not true though. Google bought a company whose main product was a Go machine, on the idea that potentially those smart people could do useful work also. Or just as PR cover for their unrelated AI work.

[eli_gottlieb]

The problem with the AI Effect is that people keep expecting solving one toy problem or another to give us some insight into how to break Moravec's paradox for the general case. Statistical learning, especially deep learning, have been massive advances precisely because they at least allow us to break the paradox for specific problems, where we happen to have large datasets.

[dsacco]

Ah, just like mathematics. Once proved, a result is trivial, but before it’s proved, a conjecture is a hard problem.

[marcosdumay]

Any game is a PR stunt, difficulty does not matter. It's not about how hard it is to solve, but on how many places you can apply them.

[argonaut]

This is one of those catchy but actually unverified memes that gets endlessly propagated.

"Nevertheless, I believe that a world-champion-level Go machine can be built within 10 years" - Feng-Hsiung Hsu, 2007 (researcher who worked on Deep Blue). https://spectrum.ieee.org/computing/software/cracking-go

[lkozma]

Yes, I think von Neumann made this claim somewhere already that AI is a moving target, because once some task is automated it doesn't seem intelligent anymore.

[mirashii]

> Chess and go don't drive innovation, they are just a side effect of real innovation.

I'm not certain this is true. Take OpenAI as a potential counterexample. While not chess or go (at least, at the moment), they are likely to be considered innovating while still working on problems that would be considered roughly equivalent. Much of the field that has been called AI for a long time (not the current deep learning approaches) were pioneered by working on chess and go. It may well be the case that proving that a new class of techniques work on these well studied games is the first step in the innovation process, where those techniques are taken and applied to other problems.

[yters]

It'd only be innovative if they'd discovered a general approach that applies to many problems without tweaking, and even better if it learned from a comparable size problem set as humans do. As it is, even something as generic as AlphaGo Zero is highly customized for the particular problem domain, and requires millions of games.

[forgot-my-pw]

The improvements from AlphaGo -> AG Master -> AG Zero is by adding more generalization techniques and rely less on human intervention/data. AlphaGo Zero learns only from self training.

AGZ probably can be retrained to other board games, but the hardware cost to train is quite expensive. The estimated cost to train AGZ (for 40 days?) was $25M.

[yters]

The AGZ algorithm is picked particularly for the sort of game that Go is.

[jorgemf]

Be aware that a lot of problems can be transformed with a linear transformation into basic and well know problems like SAT [1]. This automatically means that a lot of problems can be solved with the same algorithm. Using problems as chess or go is more about fun when doing research that other thing.

[1] https://en.wikipedia.org/wiki/Boolean_satisfiability_problem