It seems like we'd have to have been extremely lucky for the one Einstein to have been born where he was and not as a subsistence farmer if there were zero potential Einsteins among the subsistence farmer population.
We were extremely lucky, yes. Not to mention he was Jewish and being born, say, even just 15 years later than he was, could've meant he'd have ended up in concentration camp and we'd still have no theory of relativity. That's quite possible.
From a Bayesian perspective, I think we should consider the luck required for this hypothesis as a strike against it. In other words, if the hypothesis contends that observing an Einstein was very unlikely, then the fact that we did observe an Einstein is strong evidence against it.