[whatever1]

We do have standard benchmarks in the field. Hans Mittelman maintains a library. No idea why they did not bother to run them.

https://plato.asu.edu/guide.html

Their website has just 3 cherry picked instances and claim complete dominance.

[ge0ffrey]

We run the Mittelman VRPLib benchmarks at Timefold (and beat other open source solvers like or-tools in 95%+ of the X datasets).

But they are not representive of the real world, at all.

The Mittelman VRPLib benchmarks have only 1-2 constraints. Skills? No need. Working hours? Unlimited. Maps integretion? Cars can fly and the earth is a flat Euclidean space.

Any VRP algorithm optimized for the vrplib datasets is overfitted and not the best one in reality.

Take HGS for instance. Brilliant for CVRPTW. Crumbles to dust in field service routing for telco operations etc.

[whatever1]

That is fair, which is why you should not over-index on academic benchmarks. But a frontier solver should be within a small margin from the competitors in these benchmarks. If you are 10x slower, there is no way you can catch up (at least not without a ton of work from the practitioner side).