It does just move supply/demand around, that's the waterbed effect.
If you remove buy-to-let investors, then buying a home becomes easier, but not necessarily any cheaper, because the rental supply (from buy-to-let) drops and therefore the prices of rentals go up, thereby increasing the demand for buying a home, countering the lower demand from buy-to-let investors. Demand has been moved around, the shortages are still as big, and there's no significant effect on prices.
If you remove buy-to-let investors, then buying a home becomes easier, but not necessarily any cheaper, because the rental supply (from buy-to-let) drops and therefore the prices of rentals go up, thereby increasing the demand for buying a home, countering the lower demand from buy-to-let investors. Demand has been moved around, the shortages are still as big, and there's no significant effect on prices.