[jchw]

I would hope so! Otherwise the Pareto front wouldn't have moved in ~20 years which would be rather hard to believe.

However, at the same time, h265, h266 and AV1 are all vastly more complicated, and will have less broad hardware acceleration support than h264, which, again, even if it is not optimally efficient use of compute, just doesn't require a whole lot by modern standards.

So I'd argue there is really no reason to be rushing away from h264 unless you have a compelling reason. There are some obvious compelling reasons in some cases; nobody is going to be terribly surprised that an entity like Netflix is eager to switch to codecs that will save bandwidth, because at their scale saving bytes definitely adds up, bonus points if it can increase the quality at the same time.

On the other hand, though, unless you are absolutely sure you can switch to only new codecs, you will probably want to keep some h264 encodings around to act as a fallback baseline for legacy devices. And thus, you also have to take into account the complexity brought on by needing to store and maintain multiple encodings - in many use cases, like simple <video>s thrown into website backgrounds, I can see just eating the extra bandwidth costs and keeping it all h264 as a valid strategy.

[duskwuff]

Once the patents expire, I expect to see H.264 become the JPEG of video codecs - it may not be the most efficient format available, but it's widely supported, relatively easy to encode/decode, and good enough quality for many use cases.

(MP3 is in a similar place. AAC and Opus are technically superior, but everything supports MP3.)

[ksec]

>(MP3 is in a similar place. AAC and Opus are technically superior, but everything supports MP3.)

Actually MP3 is more like the H.263 / Divx here. AAC-LC is more like H.264. The difference in support of AAC-LC and MP3 is near zero, both in terms of hardware and software. While AAC-LC is vastly more superior. AAC-LC has been declared patent free ( all patent expired ) by Redhat for 9 years already.

The only part that is not perfect is there isn't a decent open source AAC-LC encoder. While you could use QAAC using Apple's encoder in iTunes. It would still have been better if there was an open source version. Luckily we just have a new AAC encoder with FFmpeg, it is doing great for and although sill not Apple's quality. But hopefully improve in the future.

Or Apple could have just open source the god damn thing.

[jchw]

I was under the impression that the new FFmpeg nmr codec indeed outperforms the Apple one in every bitrate with Google's new Zimtohrli benchmark as well as ViSQOL. That would suggest Apple doesn't really have to open source anything, we should be good?

Still, I don't see the point. Unlike video, audio decoding is so cheap you can practically do it on software even on fairly constrained devices. So really only in places like Bluetooth headphones where codec complexity can have a direct impact on battery life does it actually factor in. I say for most purposes people should just be going Opus today: it is far ahead of basically anything else. In most cases, for old devices you can simply ship software codecs instead.

Hell, Wikipedia ships video software codecs in the form of ogv.js, mainly because a lot of Apple devices wouldn't expose VP8/VP9 support even on SoCs that had hardware support for it, and that works surprisingly well. So for something like Opus, obviously it is trivial even on older hardware, even in the confines of a web browser. (And of course, Wikipedia uses it for Opus as well, but my understanding is this is only necessary for 1-2% of Internet traffic since most devices/browsers support Opus these days)

That may leave some niches where AAC-LC is still a reasonable choice, like maybe old PMPs. However, I reckon that there are probably a lot of older PMPs that in fact, don't natively support AAC. For example, the trusty Sansa Clip+ from 2009 doesn't have AAC support. If you were to install third-party software such as Rockbox, well, then you'd have Opus support.

So for audio, I think barring any specific reason not to, the meta is to basically always choose Opus.

[ksec]

>I was under the impression.....

Yes. But subjective testing by IgorC ( of HydrogenAudio ) shows it wasn't as good as Apple. It was a lot better than previous FFMPEG though. I believe we will have a few more listening test results coming out soon.

>That may leave some niches...

iPod has over 80%+ of portable music players market share. And almost all the others shipped at the time, from Creative to other Chinese brands, car audio, TV and media player has AAC support. AAC-LC hit a sweet spot of near same MP3 hardware support while being so much better. 320Kbps MP3 still don't compete with 256Kbps AAC-LC on Apple Encoder. On the other hand, there are only a few samples where 256Kbps AAC-LC isn't transparent, barely noticeable difference but no artefacts compared to Opus in professional listening test, 95% of people wouldn't even notice. Opus do shines at 128Kbps to 160Kbps range where AAC-LC don't compete ( or dare I say wasn't even tuned ), but loses nearly all hardware compatibility.

If we were streaming, and storing music or audio tracks with 128Kbps then Opus would have had strong advantage. But we live in a world where we could even stream uncompressed audio files with plenty of bandwidth left. The file size of audio is so small it makes it negligible in consumer settings unless you are Youtube. ( I would still prefer a 256Kbps audio codec, for me it was instantly noticeable. I think Youtube at one point switched for premium but decided to switch it back )

For web then I guess there is an argument for Opus. But I keep saying the same with Video Codec as well. There is a whole world of Video and Audio Codec usage outside the web and PC / Smartphone. We could have used AAC and called it a day. And it is not just patent unencumbered, it is patent free.

If I was for a new Audio Codec, I just wish it could achieve what AAC-LC 256Kbps did with 128kbps. i.e I take quality first over potential bitrate savings. But the half bitrate marketing hasn't been true since MP3 Pro, HE-AAC, HEVC and VVC for "low bitrate". Neither HEVC or VVC could achieve what AVC did with 2Mbps in 1Mbps. VVC being very close, I would guess FVC / H.267 will finally be it, but at what complexity cost? We have hit sort of limit of compression. Both in Video and Audio. And today we have very different set of trade offs between file size, connection bandwidth, compatibility and encoding / decoding complexity then what we did 10-20 years ago. And I would choose compatibility + slightly larger file size for its simplicity.

[jchw]

> Yes. But subjective testing by IgorC ( of HydrogenAudio ) shows it wasn't as good as Apple. It was a lot better than previous FFMPEG though. I believe we will have a few more listening test results coming out soon.

I am not claiming that objective tests are the only tests that matter, but I am guessing Lynne tweaked the codec to win in both objective measures and Lynne's own subjective tests. I don't think tweaking it to do better on more subjective tests would be impossible though, just needs more work, probably. What I'm saying is, at least as far as CBR goes, I think FFmpeg's new NMR encoder is likely going to be the best soon if it isn't already.

There is the question of VBR still, though.

> iPod has over 80%+ of portable music players market share. And almost all the others shipped at the time, from Creative to other Chinese brands, car audio, TV and media player has AAC support. AAC-LC hit a sweet spot of near same MP3 hardware support while being so much better. 320Kbps MP3 still don't compete with 256Kbps AAC-LC on Apple Encoder. On the other hand, there are only a few samples where 256Kbps AAC-LC isn't transparent, barely noticeable difference but no artefacts compared to Opus in professional listening test, 95% of people wouldn't even notice. Opus do shines at 128Kbps to 160Kbps range where AAC-LC don't compete ( or dare I say wasn't even tuned ), but loses nearly all hardware compatibility.

Please note that I was talking about deploying Opus vs deploying AAC-LC, not whether you encode your personal collection to AAC-LC vs Opus or something like that. (My personal audio collection is mostly FLAC and gets Opus-transcoded on-the-fly.) I brought up my Sansa Clip+ not because PMPs are still relevant - let's be honest, they're not really relevant for anyone but us nerds - but just because it's an example of a device I personally owned that really doesn't support AAC-LC.

Still, a lot of old iPods can easily decode Opus if you run Rockbox on them as far as I know, so it's not like it isn't an option if you're a fan of PMPs still, which is pretty cool since they can be had for relatively cheap depending on what direction you go.

> If we were streaming, and storing music or audio tracks with 128Kbps then Opus would have had strong advantage. But we live in a world where we could even stream uncompressed audio files with plenty of bandwidth left. The file size of audio is so small it makes it negligible in consumer settings unless you are Youtube. ( I would still prefer a 256Kbps audio codec, for me it was instantly noticeable. I think Youtube at one point switched for premium but decided to switch it back )

> For web then I guess there is an argument for Opus. But I keep saying the same with Video Codec as well. There is a whole world of Video and Audio Codec usage outside the web and PC / Smartphone. We could have used AAC and called it a day. And it is not just patent unencumbered, it is patent free.

The argument for Opus in streaming is still strong on its own. If we pit 128kbps Opus against 256kbps AAC-LC, that's obviously a 50% reduction in bandwidth for similar quality.

Does it matter if we have so much bandwidth to spare? In my opinion, definitely. For people using metered data connections, which are quite common for mobile connections in the U.S., you can fit twice as much 128kbps Opus in your spare bandwidth versus 256kbps AAC-LC - around 1 megabyte every minute versus 2. The cheapest Google Fi plan has 15 GB of bandwidth per month - if you did nothing but listen to audio, I reckon that'd give you around 4 hours a day of 256kbps audio versus 8 hours a day of 128kbps audio. Since most people do more than just listen to audio, the actual difference this makes when data limit constrained is much bigger.

Does it matter if video bitrate trumps audio anyways? Yes, still. Consider the provider end of things. Obviously, YouTube isn't serving 256kbps AAC-LC, but if they were serving 256kbps AAC-LC to everyone, then assuming they have around 4 million people streaming YouTube at any given time, they would wind up saving at least over 100 petabytes of bandwidth per month. Which is probably not a huge amount relatively speaking, but it definitely would be worth doing. In actual reality, YouTube just simply won't serve 256kbps audio to everyone because it wouldn't make sense, so Opus just enables us to get quality we weren't going to get before. Similar calculus either way, though.

Why Opus and not AV1 then? Well, I think Opus and AV1 should be adopted especially where bandwidth is a question. YouTube, Netflix, really any big provider is going to want to adopt both. AV1 offers a similar magnitude of efficiency improvement over H.264 as Opus over AAC-LC. However, the great thing about Opus is that it is just super cheap to decode, so there are limited circumstances where we're limited by compute; the only cases where it would be relevant are in fact fixed function devices. AV1 and video in general is just a lot more computationally expensive to decode, so not having hardware acceleration would be a serious impediment, and even on the Internet, AV1 support is supposedly significantly less pervasive than Opus support, so you would need to fall back for a much larger percentage of users.

So the tradeoff calculus is different, IMO, for deploying AV1 and Opus. For deploying Opus, you will need to fall back for a very small percentage of users, and doing so is relatively inexpensive because you can literally just ship a software codec. For deploying AV1, you will need to fall back for more users and shipping a software codec isn't a serious option. Which leaves some potential reasoning for people to just stick to H.264 out of sheer simplicity, in spite of the efficiency cost.

[ksec]

>Still, a lot of old iPods can easily decode Opus if you run Rockbox on them as far as I know,

For personal usage. Yes. But I was answering in terms of wider context in consumer. They don't know anything about Rockbox.

>The argument for Opus in streaming is still strong on its own. If we pit 128kbps Opus against 256kbps AAC-LC, that's obviously a 50% reduction in bandwidth for similar quality.

It is not similar quality. Not even close.

[Dylan16807]

Well a lot of the improvements cost more computation. I wouldn't be surprised if low-computation video had only improved slightly, but it seems like it has improved a lot. The advantage of h264 is hardware adoption much more than actually being computationally cheap, it seems? Well, maybe. The thing is, I don't know how the decoders scale. Can you make a decoder for a constrained AV1/h.265 in the same hardware space or maximum CPU budget as an h.264 decoder and still see very big advantages?