[prats226]

Bias wrt ordering is a great point. What we consider structured information in this benchmark is irrespective of how its presentation (Order, format etc), it should be directly comparable. So the benchmark does that it into account.

Example is if you are only converting lets say an invoice into markdown, you can introduce bias wrt ordering etc. But if the task is to find out invoice number, total amount, number of line items with headers like price, amount, description, in that case you can compare two outputs without a lot of bias. Eg even if columns are interchanged, you will still get the same metric.

[kapitalx]

Exactly. You still have to be explicit in order to remove bias. Either by sorting the keys, or looking up specific keys. For arrays, I would say order still matters. For example when you capture a list of invoice items, you should maintain order.