[bscphil]

> I think you might be interested in TCEC which has fair fights between Stockfish and LeelaChessZero

This is pretty questionable in my judgment, actually. TCEC's GPU hardware is 4x Nvidia V100 data center class GPUs, with a pretty powerful processor to boot. A quick search suggests that ONE of these will run you close to $10k, so we're talking about an all-in system worth mid five figures.

Meanwhile, the CPU hardware is pretty dated at this point. They have 4x Intel E5-4669V4, which is from early 2016. It's not easy to find this processor for sale any more (because, again, it's old), but prices seem to run in the $750 - $1500 range if you look on places like Ebay. Meanwhile even on Ebay a V100 is likely to run you $7K+.

I don't know that it's possible to compare "performance" between GPUs and CPUs in a one to one way, but looking at cost, it seems pretty clear that you'd have to spend a lot more to get a system that allows Leela to play at the kind of level you see on TCEC.

Looking at power consumption tells a similar story. Nvidia's data sheet for the V100 shows a maximum power consumption of 250 watts per GPU, so 1000W when running at maximum load (as a chess engine is presumably likely to do). Meanwhile, Intel places the TDP of the E5-4669v4 CPU at 135 watts. Even assuming they're undershooting that by a bit, we're probably talking 600 watts for that system ... on a rather old CPU model.

I'd say it's not a fair comparison. I'm not mad about it, because at the end of the day computer chess tournaments are for entertainment. It's much better if the best neural net programs are competitive with more traditional chess engines, even if by "objective" standards they are weaker.

[TomatoTomato]

TCECs goal is to keep the ratios similar to the AlphaZero paper, not price or power or other benchmark. Increasing the hardware of one would require an increase of the hardware of the other. But, the hardware is donated, so it's hard to be too critical.

But why are we comparing used 2021 prices when these hardware weren't purchased in today's market? Especially when GPUs are 1.5-2x MSRP right now. Even very old GPU prices are insane. I recently sold a 980ti near what I purchased it new. 4669v4 MSRP was $7k, so they are not far off. The V100 is pretty dated too as it is from 2017, and doesn't have FP16 which is heavily used by Leela. For this and several other reasons, a single 3090 is actually faster than 4x v100s according to their own bechmarks[1]. A single v100 is approximately equal to a 3080 or 2080ti in performance.

Maybe you should also checkout the CCCC[2] hardware which is even stronger for both: 2x A100 vs 2x AMD EPYC 7H12

[1]: https://docs.google.com/spreadsheets/d/1lGFf6PLGmBUSMan-YP7V...

[2]: https://www.chess.com/computer-chess-championship

[cout]

I'm inclined to agree. I am more impressed by an engine that plays well in limited hardware than one that plays well on faster/more expensive hardware.

This is one of the reasons Core War was so intriguing; all the programs battling it out were running on the same hardware, each given an even slice of compute time. To win, you must then find ways to do the same amount of work in less time, while keeping your footprint small.

When the day comes (and I think it will, if our civilization lasts long enough) that a computer finally "solves" chess, it will be a momentous achievement, but ultimately boring.

[zucker42]

I think TCEC is as close to a fair fight as you're going to get. Obviously, it can't be perfect. A few caveats to your comments (which I mostly agree with). One is that it's easy to add GPUs to a high end machine than CPUs. I'm surprised a 4-socket motherboard (evidently) exists. Secondly, at the consumer level, you can use a consumer GPU with similar processing power as the datacenter GPUs for much cheaper.