Ain't that the truth? But seriously, if you want a small town with a functioning downtown that isn't ridiculously priced, you are going to have much better odds of finding something east of the Mississippi River. Because the type of town you are looking for needed to be established before the automobile became big.
Geneva, IL and its neighbor to the north (St Charles) are really, really nice. But they're not a cheap places unless you are comparing them to coastal towns. Try this: make a list of all the counties in all the eastern states. Find the county seat of each one. Look at a satellite photo of the downtown area and if it seems solid, check out the street view. If that is also promising, look at real estate prices. Narrow it down to a list of 25-30 towns. And go visit them.
The average temperature barely gets below freezing and rarely get more than a couple inches of snow.
https://www.timeanddate.com/weather/usa/urbana/climate