[JohnCarmack]

I was prepared to blend the edges, but it turned out not to be necessary. If the compression ration was increased enough that there were lots of artifacts in the low res version it might be more important.

I was originally going to put it into an mp4 file with the base stored first, so normal video players could at least play the low res version, but the Android MediaExtractor fails when presented with more than 10 tracks, so I just rolled my own trivial container file.

Peak bitrate for Henry is around 40 Mbps, so it wouldn't stream for most people. With some rearrangement of the file so each strip has a full gop continuous, instead of time interleaving all 11, the bitrate wold be cut in half, but it would still be a lot of fairly small requests, so it would call for pipelined HTTP2.

[modeless]

Ah, so all the frames for every strip are interleaved in your container, and you just read sequentially and ignore the frames you don't need? That's probably the right thing for local videos.

I had each gop in a separate file like HLS or DASH (except for the background which was a single file also containing the audio track). It's unwieldy but makes HTTP streaming a little simpler because you don't need range requests or an index.

Also, instead of bitstream hacking to stitch three strips into one, I encoded multiple strips into "pre-stitched" views. This means that every strip is encoded redundantly in multiple views, bloating the on-disk video size. But for streaming that only affects the server, not the client, and it's nice for the client to only download/decode one view at a time (plus the background) instead of three strips. Bitstream hacking to join the strips would definitely be better if it can work, though.