Horses outrun moderately trained cyclists only for a very short distance, on longer distances they fall behind runners. The main benefit of a horse is that you can carry more stuff with you.
Horses must be fed even when you don't ride them. Cars and bicycles stay where you left them and don't ask for anything. But I won't leave a bicycle parked on a street for a week.