That's a valid point. What many people don't know about the French-Indian War, for example, is that the British colonists numbered in the high hundreds of thousands (if not a million) and the French colonists were about 50.000. That's why Quebec was doomed regardless of any short term military results.
Yes, English had crazy fertility rate. At the begining of the Hundred Years’ War France had population of 17 million and Britain(excluding Ireland) had 4 million. By the time of Napoleonic wars Britain(with Ireland) had 17 million people compared with 30 million in France. English also had much higher emigration rate to colonies.
When the First World War started UK had a larger population than France.