The moral position is against wearing fancy clothes that most can’t afford. Clothes are cheap now, but a modern parallel is to take a moral position against flying private jets.
That sounds like an expression of envy masquerading as moral uprightness.
It's one thing to dress modestly[0] and to exhibit moderation (circumstance will determine what is appropriate dress for the occasion, whether it is appropriate for one's station in life and within one's means, and so on). It is another to deny others goods simply because others do not have access to them. The latter is envy and a vice.
In the case of things like Elizabethan sumptuary laws ( https://www.bl.uk/collection-items/proclamation-against-exce... ), it was definitely an old money vs new money thing. Conspicuous consumption was disruptive to the social order, and tended to force an "arms race" of spending which led to a sort of "precarious upper class" situation who could afford it only by going into debt.
Does anyone have a well sourced "origin story" for why mixed clothes might have been prohibited in Judaism?
It's one thing to dress modestly[0] and to exhibit moderation (circumstance will determine what is appropriate dress for the occasion, whether it is appropriate for one's station in life and within one's means, and so on). It is another to deny others goods simply because others do not have access to them. The latter is envy and a vice.
[0] https://www.newadvent.org/summa/3169.htm