The velocity of the bike+rider is proportional to their mass. Realistically this can range over two orders of magnitude, which can be particularly relevant in a racing context.
It sounds like there could be a 100x difference between a light guy on a light bike and a heavy guy on a heavy bike. Sounds hard to believe, so I'm intrigued too :).