[fenomas]

The simple solution is just to recognize that the Ready action lets somebody attempt to do a thing, it doesn't mean they automatically succeed.

So N people can certainly declare that they all want to pass an object around, for any value of N. But if the object would need to move at supersonic speeds for them to all succeed, then obviously one of them won't succeed. (And the subsequent people won't do anything because the trigger for their Ready action didn't occur.)