[hellisothers]

The main caveat is you have your own property to do so which will usually involve already having a dwelling on it because you can’t (really) get a loan for undeveloped land. So sure you can build cheap structures, if you already have a pile of cash for empty land or a pile of cash and income for a house/mortgage. Which isn’t to say I don’t agree with you, I keep staring at our kid’s playhouse in the backyard thinking “that’d make a nice little office…”

[AnotherNobody]

This isn't really true, you can definitely get a loan for undeveloped land. It's less straightforward than a house but a friend of mine did it easily enough by finding a lender who lends on land and getting a land mortgage, you don't even need construction plans or anything.