[myaccount9786]

I wish against 1.e4 it wouldn't always play 1...c5. Right now, it's less of a test of "what is your chess rating" and more of "how much sicilian theory do you know".

Its prediction was about 300 points low compared to actual lichess rating (1800). But online chess is very different; you see a lot of unsound openings that an engine wouldn't ever play.

[garyrob]

An 1800 Lichess rating maps to 1420 FIDE or 1270 USCF. Not sure what standard is used by Noctie. (https://chessgoals.com/rating-comparison/)

[myaccount9786]

Sure maybe the translation is wrong, but I'm saying there's bias in the test due to the fact it only plays sicilian to 1.e4. What about 1.e6 or 1.e5? Hard to estimate fide or lichess rating if you don't test those.

According to https://lichess.org/analysis#1, for blitz and rapid games played on lichess with 1.e4, 1...e5 is 33%, 1...c5 is 25% and 1...e6 is 10%. And 35% is some other nonsense.

Computer chess tournaments solve this problem by requiring the computers to play specific openings.

[MereGurudev]

I think you were just unlucky with the opening! When I'm testing it's about 60/40 1...e5 vs 1...c5. The first opening moves should be quite close to what a 1600 FIDE player plays in online blitz.

[myaccount9786]

Oh haha N=2 here, so you are probably right. Survivorship bias on my part.

I posted these stats in another thread:

According to https://lichess.org/analysis#1, for blitz and rapid games played on lichess with 1.e4, 1...e5 is 33%, 1...c5 is 25% and 1...e6 is 10%. And 35% is some other nonsense.

[myaccount9786]

Does it use the frequencies from the lichess database? That would be pretty cool if it did!