[bluGill]

That only applies if you have the people. The limit of village population is the ability to grow enough food to feed it withing walking distance of the gate. And so your village cannot get a large enough population to overcome this.

Cities can only pull off a large population if there are a lot of villages growing surplus food (which is not what the village wants - in the medieval world cities were not making luxury goods farmers wanted so farmers would want to work less), and cities needed good bulk transport for all that food. Rome was a massive city in the day with just over a million people - today that is considered a tiny city and there are many much bigger.