[Dr_Birdbrain]

There are illegal moves and there are illegal moves. There is trying to move your king five squares forward (which no amateur would ever do) and there is trying to move your King to a square controlled by an unseen piece, which can happen to somebody who is distracted or otherwise off their game.

Trying to castle through check is one that occasionally happens to me (I am rated 1800 on lichess).

[CooCooCaCha]

This is an important distinction. Anyone with chess experience would never try to move their king 5 spaces, but LLMs will do crazy things like that.

[dgfitz]

Moving your king controlled by an unrealized opponent square is simply responded to with “check” no?

[james_marks]

No, that would break the rule that one cannot move into check

[dgfitz]

Sorry yes, I meant the opponent would point it out. I’ve never played professional chess.

[umanwizard]

Sure, the opponent would point it out, just like they would presumably point it out if you played any illegal move. In serious tournament games they would probably also stop the clock, call over the arbiter, and inform him or her that you made an illegal move so you can be penalized (e.g. under FIDE rules if you make an illegal move your opponent gets 2 extra minutes on the clock).

That doesn't change that it's an illegal move.

[dgfitz]

For sure. I didn’t realize moving into check was an illegal move in the sense that I’ve only played casually and the opponent (or myself) points it out.

[umanwizard]

Yeah, "illegal move" is just a synonym for "things you're not allowed to do". There's no separate category of moves that you aren't allowed to make, but that aren't considered illegal.