This is a modern myth, that has been proven to be grossly untrue, time and again.
Buddhism was a mostly upper caste phenomenon.