[bcuccioli]

I did exactly this for a school project about a year ago:

https://github.com/bcuccioli/neural-ocr

There's a paper in there that explains the design of the system and my results, which weren't great, probably due to the small size of training data.