[chrismorgan]

What’s that like for performance and power usage? I understand normal videos can generally be entirely hardware-accelerated so that the video doesn’t even touch the CPU, and are passed straight through to the compositor. I’m guessing with this you’re stuck with only accelerating individual frames, and there’ll be more back and forth so that resource usage will probably be a fair bit higher?

An interesting and unpleasant side-effect of rendering to canvas: it bypasses video autoplay blocking.

[kixelated]

It's all hardware accelerated, assuming the VideoDecoder has hardware support for the codec. VideoFrame is available in WebGL and WebGPU as a texture or gpu-buffer. We're only rendering after a`requestAnimationFrame` callback so decoded frames may get automatically skipped based on the display frame rate.

I don't think the performance would be any worse than the <video> tag. The only exception would be browser bugs. It definitely sounds like the black bars are a browser rendering bug given it's fine when recorded.

[DaleCurtis]

Unfortunately canvas (rgb'ish) can't overlay as efficiently as <video> (yuv'ish), so there is some power cost relative to the lowest power video overlays.

It really only matters in long form content where nothing else on the page is changing though.

[chrismorgan]

> It really only matters in long form content where nothing else on the page is changing though.

Did you not just describe at least 99% of all web video?

[DaleCurtis]

If only that were true, battery usage would be much better :) Just consider the prominence of content like tiktoks/shorts/reels/etc alone.

[kixelated]

Oh and the autoplay restrictions for <video> don't apply when muted.

[chrismorgan]

Depends on your configuration. Firefox has a “block audio and video” option. Which this bypasses.