One overlooked usecase is RDP and VNC-like concepts. Right now Appetize (App streaming service) uses raster graphics. Vector would allow high-FPS streaming of Apps, making running Apps in the cloud a realistic option.
Nah, raster encoders can do a pretty great job on vector-style data, whereas vector encoders can't do anything with raster-style data. In other words what happens when you play a video or view a photograph? Are you going to auto-detect that and switch modes? You'll end up just recreating a modern video codec.
The best thing to do is just use a modern video codec and make sure it works well with text and sharp edges.
Most modern application UI can't really be encoded as vectors, there's lots of raster stuff in there plus bitmaps. Many mobile apps right now are a big pile of PNGs and/or raster filters (like drop shadows, etc). So you end up actually needing to ship scene graphs along with bitmaps.
You can certainly leverage that, but it also increases the resource demands (and thus power demands) on the client.
I thought RDP was already a bit more intelligent than "only" sending raster frames, sending things like window dimensions and such to be rendered on the client?
RDP is probably more intelligent than most people are aware.
Try remoting into a machine over RDP and playing a youtube video occupying a large portion of the screen. You may be surprised to find that it plays nearly perfectly, even over non-ideal network conditions (wifi). Try this same exercise with VNC and you will experience a frame maybe once every other second.
I am not sure exactly the heuristics involved, but RDP is certainly switching between modes of operation based on what kind of visual information is on the screen.
The best thing to do is just use a modern video codec and make sure it works well with text and sharp edges.