[poly2it]

Sorry if this sounds naive, but does it make sense to write a codec library in C/ASM considering how well Rust is progressing, especially when, as the author puts it, AV2 decoding is roughly five times more complex than AV1 decoding?

[Arodex]

The algorithms deployed in these kind of codecs take into account not only human vision and mathematical laws of information, but also nitty-gritty details of how computers work, which are optimally exploited by directly having humans write detailed assembly rather than a compiler make a best guess and effort.

[alkonaut]

Surely 100% of these low level features are availale in rust too? I understand it is a massive undertaking and builds off the previous codec(s) but writing these things by hand such as inline assembly seems to be as easy if not easier in Rust?

And as soon as you walk into concurrency territory for a complex codec like this then it seems almost impossible for humans to do correctly while retaining safety.

[rafaelmn]

Why ? If it's shared reads and scoped writes (read-only look up, output to a thread owned buffer span) concurrency seems pretty straightforward.

Rust can only prove a limited subset of correct programs to be safe, when you're doing bare metal stuff you've often not in that subsystem and drop down to unsafe. I'm guessing there's always stuff that's not perf critical and can live in Rust sandbox - so not saying no wins - but it doesn't sound like Rust is a no-brainer.

[alkonaut]

I mean all else being equal, I'd just take the ergonomics (dependency management, build-system/multi-targeting, modern language features, no h-files, ...) and run. I can always just write at the level I want (safe rust, unsafe rust, C, inline asm...).

I still think no one should in 2026 be writing a nontrivial codec or anything parsing untrusted data, in C. There's just no excuse.

The gains are re-use of skill and code. And I hope that's the reason this is continuing with C, this is basically a v2 of an existing project, not a greenfield codec, even if it's much larger.

[jbk]

Because it's 5 times more complex, you need to get the maximum performance available. Therefore more ASM than ever.

Rust does not bring more performance. Just more safety.

[alkonaut]

It brings tooling that is a LOT easier. Just things like dependency management, test running and so on is so much better in Rust than in C, even if you happen to write the exact same code because you basically write unsafe code and hand rolled assembly for many things. I think this is people using the tool they know rather than the best tool (And if you know a tool well, it might become the best tool for the job because of that). It could be because a huge chunk of existing code can be re-used. But all else being equal (existing code, existing developers don't exist) I refuse to believe a codec should ever be written in C ever again.

[LoganDark]

The safety can be worth it in certain cases. Like when handling untrusted input. And it's not just Rust: look at WUFFS for example. WUFFS can actually rival handwritten implementations in certain cases.

[xp84]

Are video codecs in the present day able to be sandboxed? In my fantasies at least I’d like the worst a malicious video file can do is cause garbage output or cause the codec to crash.

Forgive the ignorance, I have worked entirely in the abstracted layers of the stack, and mostly web.

[adgjlsfhk1]

not really. they're mostly pure assembly and sandboxing assembly isn't really a things

[nullpoint420]

yes it is. all modern operating systems sandbox assembly. that's how it works.

[LoganDark]

Windows may use virtualization-based security by default, but I'm not aware of macOS or Linux doing the same -- Apple builds security directly into the silicon such that no virtualization is required, and Linux just rawdogs everything.

Whether that counts is up to you. I suppose it's still "sandboxed" in that it runs in a less privileged context than the kernel.

[throawayonthe]

but not these cases

[bigyabai]

It really should be, though: https://en.wikipedia.org/wiki/FORCEDENTRY

[IshKebab]

I don't see why not. What makes you think this is unique?

[adgjlsfhk1]

WUFFS like approaches work better for algorithms like lz77 that are substantially bandwidth constrained. for something like a video codec, the computational intensity is much higher so you need better codegen to reach max speed

[cesarb]

> Rust does not bring more performance. Just more safety.

Though more safety can in some cases bring a bit more performance. For instance, with Rust you can often avoid "defensive copies" of objects.

[itishappy]

When writing a high performance video codec avoiding defensive copies of objects is something you want always, not just often.

C makes it easy to be fast but hard to be safe. Rust makes it easy to be safe but hard to be fast.

Also note that video codecs tend to wrap C or Rust around handcrafted ASM. Performance is king.

[cogman10]

Encoder and decoder writers frequently need extremely fine grain control over SIMD instructions in order to get good performance.

The way they weave these instructions can be very hard to express with a high level language.

Further, there's a ton of work with arrays and importantly parts of arrays. They can, for example, need to extract every other element up to 1/2 the array. Unfortunately, rust has runtime array bounds checks which make writing that sort of code slower. The compiler can elade those checks, but usually only in simple cases.

The authors would be writing a bunch of unsafe rust to get the performance they want and rust makes that more painful on purpose.

I like rust, but C/ASM really is the right choice here. This is one of the few cases where rust's safety is a major detriment.

[dcsommer]

Performance should not be priority #1. Security should be. Why do we slow down all CPUs to prevent SPECTRE attacks yet continue to write in C? As rav1d shows, the perf loss is far less to migrate from C to Rust than it is to apply SPECTRE mitigations, and adding a sandbox around a memory-unsafe codec is going to be way more expensive again than using Rust code to start.

[Const-me]

> Performance should not be priority #1. Security should be.

For a web browser, or a server in a bank, sure. For anything else, questionable.

> adding a sandbox around a memory-unsafe codec is going to be way more expensive

In modern world, overhead of strong sandboxes is surprisingly small. A nuclear but most reliable option is hardware assisted VM. On modern computers with SLAT and virtualized IO the overhead for most use cases is negligible. If you want something lighter weight, can use a multi-user nature of all modern OS kernels and isolate into a separate process with restricted permissions. Sandboxing overhead is approximately zero.

[cogman10]

> As rav1d shows

rav1d is not a full rewrite of dav1d to rust. So it really doesn't show that. It's currently C + rust + asm.

I don't think we can say anything about what this does or does not prove about the performance of safe code.

> Performance should not be priority #1. Security should be.

Entirely depends on the application. The reason rust has `unsafe` is because there's some situations where performance needs to preempt potential security problems.

[dcsommer]

Codecs are difficult and expensive to develop. Therefore they get reused in many contexts, including security critical ones. Sandboxing is shown over and over to not be a great security solution, so what this means in practice is that security-critical software that needs software decoding get pwned because software engineers don't care to prioritize it in the first place.

Why shouldn't safety be the default? If you really want to, it wouldn't be too hard to maintain a patch on top of rustc to drop the bounds checks if you want to compile object files without them.

Software decoding has a safety culture problem, and we need to talk about it.

[cogman10]

> Why shouldn't safety be the default?

Because safe code isn't fast enough to decode live video.

> If you really want to, it wouldn't be too hard to maintain a patch on top of rustc to drop the bounds checks if you want to compile object files without them.

Yeah, but then you are undermining safety in a critical way that does lead to security vulnerabilities (buffer overflow). And you are also now maintaining and requiring other devs for a project to use a custom version of rustc. That's certainly part of the reason that's simply not happened.

But another major part of it is that encoders end up with a lot of custom ASM regardless. That custom ASM is going to be where vulnerabilities end up. You don't really escape that by using rust.

If you are already abandoning where you critically need safety the most for performance, then why pick a language that additionally penalizes you for using unsafe constructs?

> Software decoding has a safety culture problem, and we need to talk about it.

Compilers and languages have an optimization problem that we need to talk about. SIMD optimizations remain a very hard thing for compilers to get right. We should talk about what it'd take to make compilers better and the reasons for why codec devs need to drop down to asm instead of using a high level compiler.

There might not be a solution to this problem, there are reasons for it.

[Dylan16807]

> Because safe code isn't fast enough to decode live video.

I strongly doubt that.

And if any implementation of AV2 can be "fast enough", then there should be no question at all that we can write "fast enough" safe decoders for every other codec. Absolutely no way safe code is inherently that much slower.

[imtringued]

What's supposed to be the big source of unsafety in codecs though? Feels like the problem here is that C developers are ruining the reputation of C with their garbage code.

Bounds checking as a source of slowdown is overrated in a niche where you're working on fixed size blocks. It feels like the C developers are getting the parts outside the ASM kernels wrong.

[cogman10]

> What's supposed to be the big source of unsafety in codecs though?

Hand written assembly. It's quite easy to accidentally start reading or manipulating a block of memory you didn't intend to when doing complex SIMD transformations.

> Bounds checking as a source of slowdown is overrated in a niche where you're working on fixed size blocks.

I think you don't really understand how codecs work. It is not uncommon for a transformation like `a = b[c[i] * 3 + offset];`. There's no way for a compiler to omit the bounds check because it can't prove the contents of `c` aren't going to exceed the bounds of `b`.

This isn't a "crappy C developer" problem. This is a "There isn't a language that does a great job at capturing high level SIMD expressions" problem.

[muhbaasu]

The ffmpeg devs have said many times in public that they routinely get speedups of 10x or more over C code. I'm not a reputable source on this myself but I highly recommend looking into their channels, mails, or posts.

[nmz]

https://youtu.be/nepKKz-MzFM&t=7195

If you can stand Lex Friedman for a bit, the VLC authors talk about why you use ASM for a video decoder instead of pure C or rust.

[IshKebab]

I don't know why you've been down-voted. It definitely isn't an optimal decision. A video codec isn't all assembly. There's plenty of plain unsafe C code. E.g. this is the first random file I clicked. It has a ton of raw C pointer stuff just begging to be exploited.

https://code.videolan.org/videolan/dav2d/-/blob/main/src/dat...

There is a project to write an AV1 decoder in Rust: Rav1d (really stretching the name here).

https://github.com/memorysafety/rav1d

They got within 5% of the performance of dav1d and held a contest to close the gap but I think I read somewhere that this wasn't achieved.

https://www.memorysafety.org/blog/rav1d-perf-bounty/

They claimed

> This is enough of a difference to be a problem for potential adopters, and, frankly, it just bothers us.

But in my opinion nobody actually cares about 5% in absolute terms. It's likely just Rust naysayers using that as an excuse.

I think the likely reason for dav2d using C is that they can reuse lots of code and infrastructure from dav1d. But I agree it would be much better if they worked on Rav2d instead (these names!). You can hardly complain about a 5% overhead if you're opting in to 5x more decoding complexity.

[skelpmargyar]

Of course any random C file is going to have pointers. Where can anything in the linked code be exploited? It seems like they're testing for bad input data with asserts to catch bugs in some functions, and properly validating bad inputs in others. Just because they're writing C doesn't mean it's vulnerable.

How can you claim nobody cares about 5%? A 5% performance increase is significant. And video decoding is not always for playback, where 5% may not matter as much.

[IshKebab]

> Where can anything in the linked code be exploited?

Difficult to tell - that's the point!

[throawayonthe]

yes it makes sense to use C/ASM here, but if you're curious, there is a rust port of dav1d named rav1d: https://github.com/memorysafety/rav1d

it's not much slower than the original C/ASM implementation (last i checked ~5%?) but that matters here

[nick__m]

It's a Rust/ASM port, look there: https://github.com/memorysafety/rav1d/blob/main/src/ext/x86/...

I am not sure if it is that much safer than the C version when raw assembly is still required.

[Thaxll]

It is much slower than 5%, there were other independent tests that put it around 20%.

[Telaneo]

Go ask FFmpeg what they're writing their encoders and decoders in.

[latexr]

That isn’t particularly helpful to someone asking a question in good faith. What others are using doesn’t clarify why they are using it. Plus, FFmpeg is itself a decade older than Rust. The OP is asking about starting a new project today.

[Telaneo]

> What others are using doesn’t clarify why they are using it.

It does if you ask them, or at least research the topic at hand.

[latexr]

Isn’t that just the same as answering “Google it”, then? We’re on a discussion forum, where matter experts visit, talking about a specific topic. If one can’t ask their questions in this highly relevant situation, where can they? The point of HN is supposed to be gratifying curiosity.

[Gigachad]

Just don't try reporting a security issue to them.

[Telaneo]

Is this a reference to this: https://news.ycombinator.com/item?id=45785291 ?

If so, FFmpeg's stance is very understandable in my opinion.

[Gigachad]

Somewhat, but somewhat not. Yes it's a very obscure format, and yes it's partially a marketing stunt from Google for their AI tools. But it's also a real bug which is exploitable on ffmpeg. And we have seen in the past that state sponsored hacking groups specifically target media decoders with obscure formats that aren't often tested or known about.

Media decoders are one of the highest risk programs since they deal with untrusted user input and are incredibly complex. So just because a large project like ffmpeg uses C, doesn't mean there isn't very good reason to consider a language like Rust for saftey reasons.

[Telaneo]

If Google want secure encoders and decoders, then they can donate money or patches. Since they don't, the clearly don't actually care all that much, or are just mooching of volunteers' goodwill.

The disadvantage in speed when using Rust is pretty obvious.[1] When it comes to video encoding and decoding, I and FFmpeg care a lot more about speed than memory safety. So those reasons have been considered and largely discounted.

[1] https://xcancel.com/FFmpeg/status/1924137645988356437 (to be fair, this is only transpiled from C, so it could probably be optimised further, but that apparently needed a 20k USD bounty to then not even happen (as far as I can tell))

[2] https://www.memorysafety.org/blog/rav1d-perf-bounty/

[MattRix]

Yes? There is 5x more code to optimize the ASM for.