Margins will shrink to the point where economies of scale prevail. If garment workers tried this, it would probably not take long before they get priced out of business.
Raw production cost only factors in slightly, there certainly is room and customer demand for boutique designs and customization options of such designs.