[AnthonyR]

Have you tried this task using an actual OCR model like Google Cloud Vision AI? I am not sure if this is what Gemini uses under the hood but multi-modal LLMs are not designed to extract text like this so it should be no surprise it's not good at it?

[yoeven]

Google Cloud Vision AI is a specialized model built on CNNs frameworks which is part of the Interfaze architecture which is an hybrid so you get best of both worlds. Google cloud vision was pretty far behind other specalized models like PaddleOCR etc anyways so if you're looking for a pure CNN, check them out.

You can find the explanation and the comparison in the article, which we benchmarked pure CNN models, pure LLM models and a hybrid architecture like ours.

[schanz]

I don't think I've tried Google Cloud Vision on that particular image, no. In my experience, based on some tests from a year ago or so, Azure Document Intelligence impressed me the most in terms of OCR - out of the big three players: GCP, AWS and Azure.

I should retry the experiment because there has been a lot of progress since then and I could imagine that GCP improved there vision models since then.