[tagrun]

> They have perfectly great existing hardware decoder offloading APIs via the various OS' native APIs for videos

One vendor specific API, not "various OS' native APIs".

Firefox currently supports hardware video decoding with Intel's vendor specific VA-API only on Linux, which is not supported by NVIDIA. (A third-party VA-API to NVDEC translation layer for Linux does exist on GitHub, nvidia-vaapi-driver, but it's not yet reliable as the officially supported VDPAU or NVDEC, and is not included in official linux package repositories.)

Intel has VA-API, AMD has AMF, and NVIDIA has VDPAU which is being replaced by NVDEC/NVENC.

The idea behind Vulkan Video Extensions is to have a vendor independent and cross-platform video API.

[mikepavone]

> No, Firefox currently supports hardware video decoding with Intel's vendor specific VA-API only, which is not supported by NVIDIA.

Intel is behind VA-API originally, but I don't think it's fair to say it's a vendor specific API anymore. It's supported by the open source drivers for GPUs from all 3 vendors. It's just that the open source drivers for Nvidia cards are not very practical and the proprietary drivers only support vdpau and nvdec/nvenc

[tagrun]

You can make the same argument for VDPAU. AMD officially supports it, and there is an unofficial translation layer with limited capabilities for some Intel GPUs. Is VDPAU not a vendor specific API anymore then?

Intel, AMD and NVIDIA have their own vendor-specific video APIs, and even when they provide official support for the API of another vendor, it tends to expose a limited subset of the full functionality (like the list of available codecs and encoding features).

You are free to call these vendor specific APIs for what they are or something else, but the reality has been that there is no single video API officially supported by Intel, AMD and NVIDIA. This changed with Vulkan Video.

But Vulkan Video isn't just about desktop: mobile devices, Raspberry Pi, etc. are expected to get on board with it eventually, just like they did with Vulkan.

> It's supported by the open source drivers for GPUs from all 3 vendors.

Which 3 vendors are you referring to? Intel, AMD, and who?

> It's just that the open source drivers for Nvidia cards are not very practical and the proprietary drivers only support vdpau and nvdec/nvenc

Why are you bringing up open source drivers, and what is not practical? Both official open source drivers (open-gpu-kernel-module) and unofficial open source drivers (nouveau, through binary firmware) support VDPAU. However, NVIDIA's drivers (open source or binary) does not support VA-API.

[mikepavone]

Nouveau supports va-api on Nvidia. Nouveau is not supported by Nvidia of course.

[tagrun]

What of it? So does nvidia-vaapi-driver. They're third party projects, that's very different from "supported by the vendor", and doesn't change the fact that NVIDIA as a vendor offers no support for VA-API.

By the way, nouveau's support is currently limited and not useful: https://nouveau.freedesktop.org/VideoAcceleration.html see Video engine support status table, only old GPUs and no H.265 or AV1 support.

[mikepavone]

> What of it?

It was an answer to this question specifically

> Which 3 vendors are you referring to? Intel, AMD, and who?

I either missed some of the other text in your post or it was added after I started to reply.

> Both official open source drivers (open-gpu-kernel-module)

This is not remotely close to being a complete graphics driver. Most of a GPU driver on Linux is in userspace and there is no official open source user space component.

> Why are you bringing up open source drivers, and what is not practical?

nouveau has never been practical for serious use due to poor performance and mediocre hardware support (as you noted). open-gpu-kernel-module is only practical when paired with a proprietary userspace driver.

Anyway, my original point in all this is that describing VA-API as an Intel vendor specific API is unfair given it has been well supported on AMD GPUs for a long time now and on nouveau it's supported as well as VDPAU (i.e. not very well as you note). I did not intend to imply that it was universal. I didn't even intend to imply that VDPAU is a vendor specific API (though as a decode-only API it's not really a complete replacement).

Intel tried to make va-api the standard for hardware encode and decode on Linux, Nvidia tried to make VDPAU the standard for hardware decode on Linux. Neither was entirely successful. By contrast, NVENC/NVDEC, AMF and the Intel Media SDK (and whatever they replaced this with) never had such ambitions.

[account42]

Right, this is yet another instance of Nvidia not wanting to play nice with the other kids.

[kllrnohj]

> One vendor specific API, not "various OS' native APIs".

Incorrect. Firefox uses Windows Media Foundation, which is cross-vendor, on Windows. It uses MediaCodec on Android which is again cross-vendor. Presumably it uses whatever iOS' equivalent is as well.

It only uses VA-API on a single OS, Linux, and that's probably more a reflection on the media qualities (or lack thereof) of Linux as a whole. Maybe Vulkan video extensions will be the savior on Linux. Or maybe it won't because it won't be anyone's focus of investment since it's largely a Linux-only problem in the first place.

[tagrun]

What is "incorrect"? The full sentence that you conveniently chose to cut in the middle before quoting (apparently to fit into some pessimistic forecast about the significance of Linux desktop) reads

> Firefox currently supports hardware video decoding with Intel's vendor specific VA-API only on Linux, which is not supported by NVIDIA.

(emphasis added)

You further wrote:

> Firefox uses Windows Media Foundation, which is cross-vendor, on Windows. It uses MediaCodec on Android which is again cross-vendor.

And? None of those APIs are cross-platform. Vulkan Video will eventually allow developers (including Firefox developers) to write a single code path for video to cover a wide range of platforms and vendors (likely with the exception of walled gardens like Apple-land, although someone might find a way to support like via a wrapper like MoltenVk for Vulkan).

[Dylan16807]

> The full sentence that you conveniently chose to cut in the middle before quoting (apparently to fit into some pessimistic forecast about the significance of Linux desktop) reads

What are you talking about? They didn't quote that sentence at all, and didn't cut in the middle of the sentence they quoted.

> And? None of those APIs are cross-platform.

Your original objection, the thing that got quoted, was about whether things are cross-vendor. That question is completely unrelated to whether things are cross-platform.

[tagrun]

> What are you talking about? They didn't quote that sentence at all, and didn't cut in the middle of the sentence they quoted.

Obviously, I meant to say statement, not sentence, but I can't edit it anymore.

[tagrun]

My original statement above about what the point of Vulkan Video is

> The idea behind Vulkan Video Extensions is to have a vendor independent and cross-platform video API.

(emphasis added)

[Dylan16807]

You did say that, but it's not the part of your post they were responding to.

[tagrun]

> You did say that, but it's not the part of your post they were responding to.

So if someone criticizes a portion of your statement which is already countered by your original full statement, you're not allowed to remind your full statement. What kind of logic is that?

My original post says the point of Vulkan Video is it will be cross-platform and cross-vendor. And gives one example of cross-vendor side of things on Linux.

Someone criticizes me by essentially saying "you are incorrect, that's only on Linux. Windows, Android and iOS have their own video APIs...". This "correction" is incorrect because I already said on Linux, and it goes on to actually reinforce the post that he is responding to by highlighting cross-platform side, which also is in the post he is responding to.

So, if you look at the full conversation, the criticism is self-contradictory. This is what I'm pointing out, but you are implying I'm not allowed to do that.

I disagree. When you fragment a statement in a way that changes its meaning and make a straw man out of it, people are justified in responding to it.

[Dalewyn]

I'm always annoyed how any Linux media player or encoder needs to bring its own entire media operating system, down to each individual nut and bolt.

On Windows there's Windows Media Foundation and DirectShow that centrally manage everything and also support the "individual nut and bolt" approach. Android has its own central thing (MediaCodec?) that must be used. MacOS and iOS presumably have their own central manager (Quicktime?) too.

But Linux? It doesn't serve as an operating system for media. It's tremendously inconvenient as an admin/user rather than an evangelist.

[shmerl]

You don't need to implement every nut and bolt in the application. Lot's of useful things can do the heavy lifting (Pipewire, ffmpeg, libplacebo, Mesa and so on). Linux isn't after calling it all using some uniform "DirectFoo" naming scheme, but tools are there.

Comparison is also invalid. Linux as a whole (not the kernel but OS experience) isn't controlled by some Big Brother who decides what and how it's done single mindedly. So such kind of composite result is somewhat expected.

[pjmlp]

Hence why it will never be embraced by desktop application developers, and Electron it is.

[shmerl]

Yeah, keep complaining about everything not being proprietary enough, while everyone who needs just makes it work (OBS, mpv, etc.).

[pjmlp]

Those 2% will appreciate their efforts.

[account42]

Considering how user hostile most app developers are, I don't miss them.

[account42]

Install ffmpeg and you have all the codec support you need. How is this a real problem?

Yeah, binary software will have to ship its own copy of ffmpeg... This isn't unique to media codecs though.

[arp242]

And when someone installs some obscure or outdated and vulnerable codec on these systems, it's then automatically exposed to all sorts of applications to exploit. Maybe Windows sandboxes that these days(?) It was definitely a problem in the past.

No perfect solutions here; both "system-wide codecs" and "every application brings their own codecs" have their own up- and downsides.

Besides, with ffmpeg and gstreamer the system-wide codecs paradigm also works on Linux.

This is one of those "it's different but it doesn't really matter much" type of things. Most people "just" install vlc or mpv or whatnot and things will "just work" for them, not really different from Windows. That it's technically slightly different is almost entirely transparent to the user.

[pjmlp]

Yeah on UNIX side, NeXTSTEP, Irix, Solaris had their own thing, as graphical workstation UNIXes, and were great.

Ideally that kind of thing would be part of GNOME, or KDE, but then there are those that rather keep using twm like experience, making GNU/Linux really only good for headless experiences, at least the UNIX/POSIX part is always there.

[vetinari]

AMD AMF is not open-source; only the SDK part of it is. The runtime part is closed, bundled with the "pro" drivers.

It is also intended as a multi-platform abstraction.

This makes it a no-go as a platform API. The open drivers for AMD use VA-API.