[thom]

You can checkout the exact commit of Stockfish from the paper and perform the analysis of the published games yourself. I doubt the Stockfish developers have bothered because it's an old version.

Can I just state again, because of your aggressive tone in multiple comments now, that I do believe AlphaZero is stronger, I don't believe there are real shenanigans going on, but it's _still_ sad that we can't reliably, publicly verify this stuff.

[mda]

You are the one making this claim: "You can still run the games past the exact commit of Stockfish they used and it finds blunders in its own play, so it still feels like there's a lack of transparency."

This claim implies foul play, I asked for a shred of evidence. Call me aggresive if you want, I just can't stand this kind of bullshit.

[thom]

It's not bullshit, you can download the version of Stockfish and ask it to analyse the games! I did this with scid, you can too. Around move 27 in game one is one example. I don't intend to repeat the tedious process because my curiosity is satisfied, and you're just being obnoxious.

[mda]

Did you really emulate all the conditions published in the page? I seriously doubt it. Otherwise all your claims are speculation and moot.

[thom]

It's impossible for me to know how long the engines thought on each move, so I just analyzed the games at a variety of times and depths. Stockfish 8 at two minutes per move on a machine that is slower than that used in the match finds plenty of issues. If we know the amount of time AZ thought (or indeed... had access to AZ) it'd be possible to more closely reproduce the games.

This is all either of us would have done if we were peer reviewing the paper, don't really understand the hostility about trying to reproduce a published paper.

[thom]

(And I know in the past, before reading the paper, I've assumed they were cheating this time because they made so little effort the first time around, but I'm happy to admit I was wrong).