[cillian64]

On Linux there is already an alternative to VAAPI: The Video4Linux Memory-2-memory (V4L2-M2M) API, which provides a standardised API for hardware encode and decode accelerators (and supports using DMA-bufs for copy-free decoding or using decoded frames as textures). I don't know how much support there is for AMD/nvidia/intel but it works well on the Raspberry Pi at least.

It would feel weird for Vulkan to become the standard API for encode/decode given that (as you say) video accelerators aren't necessarily tied to the GPU (although not necessarily weirder than doing it through an extension of the webcam API).

[jchw]

Truth told, Raspberry Pi is the only device I've ever used v4l2 to do video encoding with. It does seem to work well enough, but since I was using it via GStreamer, I have no idea what the API looks like. The advantage of Vulkan is that they've already got a functional and productive committee with all of the GPU vendor stakeholders to work out the design, whereas VA-API and VDPAU seem to have somewhat different designs and seem to be mostly championed by a single GPU vendor. I have a bit more confidence that the Vulkan designs for hardware accelerated video will be good for both hardware vendors and application developers (even if it is still low-level.)

In a neighboring comment thread, hrydgard states that it is very much possible to have a Vulkan driver that has only video encoding/decoding queues, which if true resolves the potential awkwardness in my opinion. That's especially relieving if true as well since Vulkan looks to be a prime contender to take up the mantle for OpenCL as well; especially believable now seeing what has been done with rusticl.

Perhaps it is still a little awkward, but Vulkan as a general hardware acceleration abstraction layer seems like it could be a very good thing. As it is, as an application developer, you're probably better off on Linux just using GStreamer or Pipewire rather than v4l2 directly, since some devices (e.g. Blackmagic Decklink) provide GStreamer integration but not v4l2 drivers. In that case, it hardly matters what API abstracts the hardware as long as it does a good job of it.