That’s a real aspect, but only one dimension. Nonetheless, it’s telling that it’s what you distill the entire problem down to. There are many other problems that make the problem hard:
a) these communities often depend on consensus decision making. Problems that require collective action get nearly impossible when the society gets too big
b) People underestimate how much they depend on greater society in a modern context
c) It’s normal for future generations to have evolving values. The original idealistic notions tend to lose their luster to those who were born into it
There are others more unique to communes, but I highlighted those because they also make it hard for the inverse libertarian ideal to work as well. Meaning it has to do as much with human nature as any ideology.