[sarosh]

The paper itself is here: https://arxiv.org/abs/2111.09259 with the key conclusion that "Examining the evolution of human concepts using probing showed that many human concepts can be accurately regressed from the AZ network after training, even though AlphaZero has never seen a human game of chess, and there is no objective function promoting human-like play or activations" and "[t]he fact that human concepts can be located even in a superhuman system trained by self-play broadens the range of systems in which we should expect to find human-understandable concepts"

[iechoz6H]

Aren't the shared concepts simply predicated on 'the rules of chess'? No surprise that an algorithm and a 'rational actor' share similar themes when both are constrained by the same rules.

[baq]

the concept of material isn't found in the rules and yet was discovered independently during self-play.

[magicalhippo]

> the concept of material isn't found in the rules

It's not spelled out directly, but it is a pretty low-hanging fruit.

After all, the rules dictate that the number of pieces on the board never increases. The rules also dictate the different kinds of moves allowed by each piece. Together it seems to me those rules invariably lead to pieces not having the same value, aka the concept of material.

[Santosh83]

We are quite literally alpha zero ourselves. We developed chess by playing ourselves. I don't get what is particularly surprising that there are not infinite ways to play good chess?

[ogogmad]

It's not a priori obvious that a machine can learn to play chess in a similar way to the way a human can.

I suppose if AGI gets achieved then some of this will seem predictable in hindsight. But we don't yet know if it's achievable.

[pmontra]

Maybe for more complex games, example: economics. But chess is pretty simple compared to seven billion people going their own ways.

On the other side, AlphaZero and all the previous Deep Mind's AIs played some very different go than human players did. Every single move was not unheard of but they were dismissed by human masters as not good. The most noticeable contribution was the total change of how to play corner invasions. By looking at game records we can draw a line between pre and post AlphaGo. Maybe go is a little bit more wide than chess and AIs have more space to find novel plays.

[alchemst]

We (humans, AlphaZero) are both distilling the rules down. Some rules are easily distilled (a queen is worth more than pawn) some are more difficult (mobility, knight vs bishop). I would expect the easily distilled rules to be discovered by both of us.

What I think would be cool is trying to distill or revise better guidelines from what AlphaZero does.

[V-2]

Human minds didn't evolve for playing chess. So there was a possibility that there are "hardware" limitations that prevent us from playng it the way it should be (give or take the inevitable error margin).

[kevinventullo]

The human-understandable concepts are there, but I think finding them is still very hard, right? I.e. it doesn’t sound like the system including AZ was able to identify and articulate the human concepts such as king safety a priori.

[iratewizard]

A game like chess is like two competing mathematical models. The human understanding of it is really just noticing patterns that exist in the model. I would hope an ML algorithm finds similar patterns.

[posterboy]

Can this be explained by the fact that the rules of chess evolved accomodating certain preferences that human players developed over the course of time?

Take a much simpler game like paper, rock scissors, where certain strategies exist as well (I hear). This should be much easier to analyze. Can somebody apply alphaZero to rock, paper, scissors, please?