[bluquark]

So as a specific example, would you say there's no excuse for not being able decode a 1GB PNG image in less than 100ms? The state of the art algorithms, in Rust and Wuff programming languages, have reached around 500MB/second decode speed on a desktop x86.

If I had specified a 1GB image with a PNG-like compression ratio and allowed you define the data format along with the algorithm, then the challenge would not be so difficult. But the moment the requirement is an image format that's actually widely used...

[_gabe_]

PNG is a compressed format, so right off the bat it would be weird to have 1GB of compressed image data. According to this[0] out of 23 million analyzed images from web pages, PNGs comprised around 14.4% of those images and averaged around 4.4KB. So this scenario isn't likely to occur in the real world, and if it did, decompressing 4.4KB would happen well under 100ms.

Secondly, PNG is an unnecessarily slow algorithm as we've found out from QOI[1]. From the results, it can encode over 300 megapixel per second in some cases. Now I'm a bit fuzzy on the math, but I believe 300 megapixels means 300 million pixels, and if we assume 16 bytes per pixel (4 bytes RGBA) than that means it can encode at speeds of up to 4 GB/s and achieve very similar compression ratios to PNG. Oh and by the way this algorithm isn't even using multithreading, so I assume it can be sped up even more.

So I wouldn't insinuate that the developers who coded the implementations to decode PNG are programming slow inefficient code, but they're stuck with a bad algorithm from the outset because of the requirements, as you've already pointed out.

I think my original statement still stands.

> There's no excuse for why any algorithm should take more than 100ms

The PNG algorithm is inherently flawed imo. We could do better. The developers who are stuck with this specific compression algorithm have no choice but to do the best with what they've got. But you can always transform the PNG image into a faster lossless format like QOI and then use that. So it's still not worthwhile to give up and say we can't do better since PNG is inherently slow.

Edit: I think my original statement almost stands. Processing 100s of GBs of data in under 100ms is definitely overestimating our current computing capacity. I would probably amend that to something like processing anything under a few GBs of data (3-5 GBs) is definitely possible in under 100ms.

[0]: https://www.pingdom.com/blog/new-facts-and-figures-about-ima...

[1]: https://qoiformat.org/benchmark/

[bluquark]

> The developers who are stuck with this specific compression algorithm have no choice but to do the best with what they've got.

Right. The point I wanted to make is that with the exception of game developers, who have a unique freedom to define their set of requirements and design a vertically integrated system to solve them, every developer faces their own version of this.

So I agree games are a useful reference point to conceptualize what's possible after an extended transition plan to migrate ecosystems to new standards. But I don't agree with the framing of "no excuses". It takes heroic efforts and wisdom to break out of ecosystem local maximums. For example, when image transcoding to newer formats has been tried, users hated how other software didn't read it correctly if they tried to save or reshare it. So with the latest attempt (JpegXL) they're aiming to gather industrywide support instead of acting unilaterally.

We should cheer orders-of-magnitude performance improvements in the rare cases when a migration can be successfully coordinated to so do, instead of being unduly negative about the normal expected case when it isn't.

[int_19h]

The PNG example is a case of a well-established standard limiting what you can do - basically, improvement is only possible by breaking compatibility.

But this is not the case for the vast majority of slow software. Nor is there any equivalent problem there. It really is, as the article describes, a case of devs not caring to put the time because the users tolerate the status quo.