[energy123]

GPT-4o is very good at some visual tasks like optical character recognition. So the selective blindness might just be what you say here -- all of its capacity is dedicated to minimizing loss on a few narrow tasks that had the most training data (like OCR). So it's not necessarily an inherent failure of the architecture to generalize, it could just be a capacity issue that will naturally be resolved with more scale.

[sushid]

Is that not just traditional OCR applied on top of LLM?

[energy123]

It's possible they have a software layer that does that. But I was assuming they don't, because the open source multimodal models don't.

[maxlamb]

No it’s not, it’s a multimodal transformer model.