Animals are lossy at converting plant calories into meat calories (varies for different kinds of livestock obviously). I don’t think it’s an obvious conclusion that more agricultural land would be required.
Did you get the impression that I am speaking for abolishing agriculture?
To answer your question, almost anything will do as feed.
Byproducts of agriculture, grasslands, processed/filtered organic waste.
Animals will thrive on rotten meat too -- famously leading to the mad cow disease.
In comparison, animal rearing is feasible in pretty desolate places as long as you can transport feed and water.
Just claiming it's a lossy energy conversion is pretty irrelevant. Not all the 'calories' here are created equal.