Home size is a factor, but lot size is a more important one. Many cities explicitly or implicitly require minimum lot sizes and setbacks that make it pointless for builders to make small starter houses.
I wouldn't even say lot size is a factor as much as simply zoning rules and regulations. there aren't a lot of high cost of living areas, with the exception of maybe places like Manhattan, that actually don't have space for more housing. and even there, they have plenty of under-utilized space zoned for other things.