Someone else tried this with GPT-2 a few months ago on algebraic notation and their engine seems to get to move 40 without blundering: https://www.reddit.com/r/slatestarcodex/comments/el87vo/a_ve...
Board state + algebraic notation might be the trick to make a strong engine.