[potatoman22]

Counter intuitively, language models aren't good with letters and string manipulation. It probably has to do with tokens being a few letters and the lack of those tasks in their dataset.