I use a WebAssembly build of FFmpeg to extract the streams and packets from the video files and as a fallback for some codecs. But most of the decoding/encoding is done with WebCodecs which in most cases has access to hardware acceleration so can be much faster than ffmpeg.