The only places where buildings go up in value is where the building has been grandfathered in and is no longer able to be built in accordance with modern zoning rules.
Which is rather common in my experience. The town adjacent to the one I just moved to has a bunch of duplexes, but if you look at the current zoning it's all single-family-only. You couldn't build much of the town as it is today, which is a shame as it's very nice.
As you might expect given this, the town has steadily lost population.
As you might expect given this, the town has steadily lost population.