[_kidlike]

interesting... I have never seen such a benchmark before, where it forces the model to use a json schema for parsing a document. Sounds a bit counter intuitive to me, but I'm not that deep in the field. I usually just "ask" the model for information that is inside an image pdf. I haven't run any sophisticated benchmarks though...

[menaerus]

No, they don't force the model to use a json schema, they simply use the model to extract the data, and then they feed that OCR result into the pipeline further to evaluate the OCR results against the ground truth, and this is where JSON schema is used, and also another model (gpt4o).

[_kidlike]

aaahhh...