[indy7500]

The way we check legality and adjust is with the character recognition. We use the second highest and third highest confidence characters. So, if you write a 5 and the model recognizes it as a 5 and then a 6 and 7, but really it's a 4, we can't help. This is because we'd add moves exponentially every time we add that many combinations

[theli0nheart]

I don't get it. If Bg5 is an illegal move then the recognition is obviously wrong and you can prune it (I would think?).

Who am I to talk though, I'm not the one programming this thing. That's just what my gut tells me. :)

[indy7500]

Don't say that, you're definitely more experienced than me :) I'll try to clarify, but I could be wrong. This is from the chess.com article on it here: https://www.chess.com/blog/ReineChess/scannable-scoresheets-...

The model actually gives us a confidence for each possibility. It might say, N-90%, K-10%, B/R/Q-0% for one of the boxes. Here, we look at how confident our character recognizer is on each character. If the PGN is invalid, then we know some character was recognized incorrectly. We look at the low-confidence characters and change them to the next highest confidence prediction, checking to see which combination of changes delivers a valid PGN. For example, consider the string of moves 1. d4 d2. Black can't play d2 on move 1! But we look at the next most likely predictions, perhaps d8 for white and d5 for black. These are the combinations:

d4 d2

d4 d5

d8 d2

d8 d5

Only the second one is valid, so we choose it and continue ahead.

If you literally wrote "d4 d2," then it's a lot less likely that the correct digit instead of 2 will be in the top 3 predictions.

[theli0nheart]

This all makes really good sense. For now I think this is a workable strategy, but I bet you can do better (take that as a challenge ;) ).

Anyways, this is really cool, and smart idea. I'll be printing these out and bringing them to my next tourney. Excited to see how it goes! How do I send you direct feedback?

[indy7500]

We currently don't store scoresheets when uploaded to our website. Would you be able to send the pictures to us at [e-mail]? You could add comments about how it went (or not, since we can just run it and see for ourselves). If you think that might be a hassle I could quickly add a feedback form on the website. Also, I'd encourage that you try it once before the tourney with the restrictions—we haven't tested on very varied handwriting. Oh, and thanks so much for offering feedback!

P.S. Did you have something in mind for improving the postprocessing / trying out different iterations? Or was it just a challenge :D

[fr0sty]

Is there a reason you don't store the images of submitted games? Acquiring real-world training data to test changes with would be very useful. Perhaps make it an opt-in feature people can select when uploading a game so people are "offering" you this information to you rather than you just "taking" it.

One way to choose between two legal candidate moves would be to look forward for the next move by the candidate pieces and see which ones make sense given. If you are trying to choose between e6 and e5, for instance, look forward and see what the next e pawn move is. Plenty of other ways to go about this, of course.

[indy7500]

No, not a clear reason other than wanted to get the product out earlier. We'll work on an opt-in, that does sound the best.

Wouldn't other pieces affect this though? For example, what if the move was e6 and the next e-pawn move was e7. What if a piece was already there, Ne7 (wrongly recognized)? I thought about slowly running through it like this, but it felt like I kept running into problems which are obvious for us to recognize but I wasn't sure how to implement it computationally