What about geostationary orbit wrt balloons? Do the balloons slow down horizontally a lot during the ascent? If not, what if we floated a rocket up most of the way and let the rocket do the rest?
Balloons float because dense air in the lower parts of atmosphere pushes them up with buoyant force. They lose the ability to float way before they would reach LEO, let alone GEO.
You've just described a ballocket. You launch a rocket from a very high altitude balloon.
You gain nothing from the height, because you still need to spend ~10km/s getting up to orbital velocity, but you don't have to push your way through the thick lower atmosphere any more. Rockets don't work very well in atmosphere, so you do get fuel savings due to more efficient engines.
I don't know if this has ever been tried for real.