[logicalmonster]

Just some random thoughts.

1) If you really want to, you can probably take out a loan and buy/build some kind of summer house with $100K savings. That's more than enough for a solid down payment and the needed extra expenses, or even buying something in cash if you're willing to sacrifice on the size/location of the property and quality of the house, depending on your desired living standards.

2) If your budget truly is already tight, perhaps going to buy a second house might not be the wisest move heading into a potentially very rough economy. Perhaps you can reduce those concerns by AirBnB'ing it, but take into account that in a down-economy excess travel is going to take a hit too.

3) Depending on many factors I can't possibly evaluate (age, health, relationship status, kids, life satisfaction and mental state, etc) an optimal middle ground might be to try and save more cash and evaluate where the economy is in the next couple of years. Being cash-rich when there's a downturn is a great spot to be in.