Hacker News new | ask | show | jobs
by amoerie 1108 days ago
As a radiology software developer: please please let JPEG XL become a thing. I sincerely hope this might make Chrome change its mind. JPEG XL is a game changer for 16 bit lossless images. The technical landscape for these kinds of images today is barren.
7 comments

As a mirrorless photographer who wants to publish photos on the web that look like they came out of a mirrorless camera I want JPEG XL. In trials I've done, AVIF works well for a throwaway splash image for a blog but compression results are not so impressive compared to JPEG or WEBP if you want the image to hold up under close inspection.

On the other hand there is something that seems almost infantile about image support in the "operating system" being pivotal. If you read a review of a new MacOS in ArsTechnica you might get the idea that 99% of an OS is about what the buttons look like but in terms of the computer science definition, image codecs are definitely a userspace thing and as a Windows or Linux user I never wait for my OS to support an image format, I just install the codec and code away.

You're using the term "OS" but you're not talking about OSes you're talking about kernels.

The OS is what comes with the computer or gets installed in the default setup if you're installing yourself. Most of it is userspace.

Distros like Arch & Gentoo allow you to build a custom OS which in doing so blurs the definition quite a lot, but ultimately when people say "macOS" that term includes quite a lot of userspace things. Bundled software is absolutely a core part of every popularly used definition of the word "OS".

As an ‘operating system’ OS X includes user space tools like the Finder and Quick Look in particular where vendor support is helpful.
also crucially, safari
Yes image codecs are a user space thing, but a Mac or iOS app probably loads images using the Swift Image or UIImage class and passes it a filename, and then these same objects get passed to the display widgets to render. If the UI toolkit that ships with the OS supports a format, it will just work with most all of the apps that use the system UI toolkit.
There’s just not a lot left besides user experience and appearance for an OS to do.

Of course kennels and operating systems are highly complex and represent a ton of work… but what is left to impress with a press release?

My OS does everything I can imagine needing it to and the only thing I can imagine wanting it to do going forward is more or less just maintenance (UX/UI excluded)

Dumb question, but why is OS or browser support necessary? Couldn't an HTML canvas element and some JS that can parse the file format display any kind of image that you might want?
On MacOS/iOS the ImageIO framework handles encoding and decoding images. Adding support for your image format to that framework means everything else just works once the image is decoded.

Another framework, CoreImage, provides generic operations to filter and transform images decoded by ImageIO. This provides hardware acceleration and other good stuff that make things like CSS transforms super fast and efficient.

While it can be done in JavaScript it would be much much slower, less efficient, and kill your battery.

There's a JS/WASM polyfill for JXL, but it involves some fragile hacks like MutationObserver and canvas writes, has worse performance than native code, and tends to crash Chrome.
Why would MutationObserver be necessary to display a static image in a <canvas>?
Yes, if you don't care for loading speed, battery life, and the fans blazing...
Canvas has hard caps on the size it can be as well and when resizing an image to a fixed size, there is a lot of hard edges as you need to implement your own smoothing. Image tag is more flexible and versatile.
HDR and progression are difficult to do right now. Possible interplay with CPU/GPU in rendering is difficult. JS is executed later than html is parsed.
A canvas element is not as performant as an image element.
When I am serious about images it is all about high performance: upgrading to better data compression (1) speeds up the network and server and reduces (2) network and (3) storage costs. (4) Decompression of the image is another bottleneck and a soft decoder is itself something to download and compile locally.

If you use a high-powered device you might not appreciate that performance of a low-end Android device hasn't really improved since 2017. I test with an Android Go device that does pretty well if you feed it scaled images but would struggle with the soft decoder.

The native implementation of the decompressor can be much better than one in WebAssembly in that it can use SIMD units on the CPU and many other tricks, including special purpose hardware. That's why Apple is so keen to say that an image format is now supported in the OS, because they codesign the hardware, the OS and the file formats to take advantage of all that.

Why down-mod this straightforward question?

Come on, people.

> publish photos on the web that look like they came out of a mirrorless camera I want JPEG XL

Any comparision photos out there?

>As a mirrorless photographer who wants to publish photos on the web that look like they came out of a mirrorless camera I want JPEG XL.

As if people would have the screen to properly watch them? Perhaps a handful of high-end laptop owners... Except if those are the intended audience.

>image codecs are definitely a userspace thing and as a Windows or Linux user I never wait for my OS to support an image format, I just install the codec and code away.

That's how you get half the apps supporting them (if you're lucky) and others not, and generally a hell of a time...

There are hundreds of millions of recent iPhones with outstanding OLED displays out there.
Is your audience iPhone users? Or is it an progressive enhacement style thing?

(Btw, there's nothing about this that's specific to mirrorless and wouldn't also apply to a DSLR with big dynamic range).

I'll go on the record and say that, unlike audio, where I'd be very concerned about the quality of speakers, I am not so concerned about the screen quality where my photographs are displayed. That is, even my Android Go device is "good enough" in that if you really need to see more detail you can zoom in.

A better comparison with audio would be the terrible quality of a 64 kbps Mp3 file which is going to sound awful on the cheapest bluetooth speakers or a $5000 set of speakers compared to 128 kbps Mp3 which sounds OK superficially but falls apart when compared to the source CD, or higher bit rates which are close to transparent. Many images for the web are highly compressed but nobody really notices.

My current boggle w/ screens is actually what to do with "high color gamut" screens like the one on my iPad. I am into red-cyan anaglyph images where the most important thing is getting separation between the left and right channels.

https://en.wikipedia.org/wiki/Anaglyph_3D

The green in a high color gamut display is more saturated than the sRGB primary so when you ask for sRGB green you get some red and blue mixed in which is an absolute disaster for a stereogram.

The answer to this is to master a high color gamut image just for those devices and serve everyone else an sRGB. I will get around to it one day but it's been a higher priority for me to deal with the same problem in print, where the green on my printer is very saturated but also very dark and the color management system blends in some red to make it brighter. Turning off color management works but really I should be blending in some red into green areas of the left image because that will make the colors closer to the original and also help with stereo imaging by reducing the difference in brightness between the left and right channels.

> Many images for the web are highly compressed but nobody really notices.

Or they do but they just assume that's the way things are. They don't know why an image that's been reposted from Facebook to Twitter to Instagram to Reddit to Imgur and back a dozen times has artifacting and color banding, and as long as they can still get the gist of the image, it doesn't matter to them. Besides, it's not like they can just ask the platform to magically fix the image, so they just don't think about it.

In many cases, the original high quality source has been lost to 404s and unpaid webhosting bills. The original is probably still on someone's hard drive, but it's unlikely to be remembered, much less surface again, so compressed messes are all that's left.

With JPEG XL, many of those compressed messes would be much closer to the original image due to its extremely strong generational loss resilience (see the webm attachment to [0]).

JPEG XL is a massive step forward for images on the web for a number of reasons, but it has especially massive benefits when it comes to preserving image quality across the web. And this is especially true when you consider that you can reversibly reencode the billions of existing JPEGs on people's drives and on the web as JPEG-XLs losslessly for 20+% space savings [0].

[0]: https://bugs.chromium.org/p/chromium/issues/detail?id=117805...

Ex-radiology software developer here, and seconded! J2K is good, but here are some advantages of JXL in the context of medical imaging:

- Better compression ratios

- Ability to modify effort of lossless compression (good for real time transcoding)

- Multi-threaded encode and decode

- Far superior progressive decoding (great for low bandwidth scenarios, just stream in the lossless quality over time)

> As a radiology software developer: please please let JPEG XL become a thing.

If things with Apple go well and JPEG XL is supported natively on macOS, iOS and iPadOS this fall, it would be on its way to becoming a thing.

After all, 2 billion devices isn't nothin’.

> I sincerely hope this might make Chrome change its mind.

I take this to suggest that Chrome actively decided against implementing JPEG XL? Did they consider supporting it and reject support for it, or has it simply not been prioritized, but still might be prioritized one day?

If the decision was intentional: did they state a reason why?

This is the terrible explanation they gave for dropping support:

https://bugs.chromium.org/p/chromium/issues/detail?id=117805...

"Thank you everyone for your comments and feedback regarding JPEG XL. We will be removing the JPEG XL code and flag from Chromium for the following reasons:

- Experimental flags and code should not remain indefinitely

- There is not enough interest from the entire ecosystem to continue experimenting with JPEG XL

- The new image format does not bring sufficient incremental benefits over existing formats to warrant enabling it by default

- By removing the flag and the code in M110, it reduces the maintenance burden and allows us to focus on improving existing formats in Chrome"

The issue on the Chromium tracker is now one of the most-starred and most-commented-on of all time because people from all over came to tell Google that they're insane, from Intel to Adobe to Facebook to Krita to Cloudinary to Shopify to Serif/Affinity to the VESA DisplayHDR Chairman.

It may also be worth noting that the author of commit to remove support from Chromium appears to be a WebP co-author, having given talks about WebP and being the primary contributor to libwebp.

https://chromium-review.googlesource.com/c/chromium/src/+/40...

https://chromium.googlesource.com/webm/libwebp/+log

WebP and libwebp/cwebp is such a clusterfuck. Lossless mode isn't even visually lossless because cwebp doesn't understand how PNGs encode color space information. Default lossy settings are extremely garbage for darker (parts of images) images - a problem that has plagued video codecs (which is what webp is based on) for a long time. Animated webp is ridiculously inefficient compared to webm and really shouldn't be a thing at all in favor of allowing silent+looping videos in image tags. And of course initial webp browser implementations don't even support animated webp but still claim support for image/webp so you can't even do progressive enhancement from gif to animated webp without hardcoding which browsers support what.

At this point it should really just be deprecated in favor of JPEG XL. And let's skip AVIF completely please.

100% agree. I'd say JXL is going to be the closest thing we get to a universal image format for quite some time. Almost completely superior to JPEG/PNG/GIF/WebP/etc. and clearly superior to AVIF in basically every way except very low bitrates and animation (which... just use HTML5 video with AV1 webms, what are you even doing?).

Oh, and adoption rates, but considering JXL's standard was finalized less than a year ago and it's already gotten support in so many things and from so many large companies, I really don't see any way that it fails other than Google abusing their monopolistic position in the browser engine space. The people arguing against adding support for a brand new codec because it doesn't magically have 100% support is circular reasoning and feels very disingenuous considering WebP and AVIF were never held to the same standard.

>At this point it should really just be deprecated in favor of JPEG XL. And let's skip AVIF completely please.

You say that now, try doing that between 2020 all the way to 2022 you get attacked by AOM Strike Force.

They did all the work required to implement it, sat around for a few months, and then removed it, before anyone else even noticed it was there / had a chance to start integrating it.

My personal conspiracy theory is that some Google engineer who works on Chrome, came up with "JPEG XL support" as a feature they could work on, pushed it through to prod, patted themselves on the back, and forgot about it; but this was all done without first getting sign-off from whoever at Google is trying to push for WebP to be a thing. When that person or group noticed "Chrome now supports JPEG XL", they got that support ripped out.

The work wasn't done by anyone on the Chrome team. Someone at Google Research Zurich (I don't recall who) wrote the patches and submitted them. Actually, I think that they've even kept the patches up to date with newer Chrome releases so in theory it would be easy to re-introduce JXL into Chrome if there is a will to do it...
The number of people in this thread pretending they have more expertise about this, and spreading random conspiracy theories about WebP, is impressive.

Actually, Google employs 2 of the 3 main authors of the JPEG XL spec, and the main contributors to libjxl.

Also, it wasn't a few months, and others had explicitly said no.

You can argue it was a dumb decision, but like, can we at least get facts straight instead of making up random stories?

There people in this thread arguing they have no idea what they are doing and have no expertise, while they simultaneously employ the spec authors to work on it and are one of the two primary contributors to the reference library.

A bit silly and incongruous.

Just because someone does something you don't like doesn't make them stupid or wrong. You'd be much better off if you would gather facts first, listen to the perspectives of others, and then respond.

> Actually, Google employs 2 of the 3 main authors of the JPEG XL spec, and the main contributors to libjxl.

> There people in this thread arguing they have no idea what they are doing and have no expertise, while they simultaneously employ the spec authors to work on it and are one of the two primary contributors to the reference library.

Google is not a monolith. The JXL people are at Google Research Zurich, while the people who decided to not include JXL in Chrome are members of the Chrome team. The Chrome team obviously does not employ the people over in Zurich, nor do they presumably control what they work on.

I am sorry but Google's rationale and comparison [1] between JpegXL and WebP is even more impressive.

And much like Microsoft, I will put any research arm of a company as a separate company.

[1] http://storage.googleapis.com/avif-comparison/index.html

Companies participating in standards bodies while actively sabotaging the standards is nothing new.
it was also only ever hidden behind a flag
The Googlers behind webp have more clout than the Googlers behind Jpeg XL.

JPEG XL has gained broader industry interest than any of the previous attempts to replace JPEG.

I think someone is very hurt that their web p pet project has earned more hate than love

>If the decision was intentional: did they state a reason why?

Yes It was intentional.

They stated AVIF is as good if not better than JPEG XL.

Can you explain to the rest of us what about JPEG XL applies to that use case? If you asked me to store lossless, 16-bit (per channel, or monochrome, I am assuming) images I would suggest TIFF.
I think they want better compression. JXL has excellent lossless compression.
TIFF is a container, it can have payload with different compression schemes, lossy or lossless. Including JXL.
TIFF can be used as a container. It also comes with its own (relatively shitty) compression methods which is probably what you'd end up using if you wanted 16-bit TIFFs today.

And TIFF as a container for JXL would no less require JXL adoption than plain JXL.

I am suuuper ignorant on the subject, but is this something that could replace DICOM?
Not an expert either by far, but DICOM isn't just images, it's a much larger data structure and following protocol, which deals with the transmission of the data between devices as well. The files are just one part of it. It also supports more dimensions to the data frames other than 2D. The pixel data in DICOM itself uses JPEG however (among other potential pixel compression methods), and it is possible that JPEG XL rolled into DICOM could allow for benefits.
No, DICOM is a container format that can contain JPEG XL image data
Ah, I see. What does the contain provide other than image data?
Lots of metadata about the image scan. Patient name/id, etc. and often physics related to the image scan such as the voxel size, units, properties of the imaging equipment, etc. DICOM is also used for other data that isn’t directly related to the visual image such as segmentation of the image, radiotherapy planning info, and so on. There are hundreds of dicom tags (the metadata entries) that can be in the dataset.
Does it really matter? These days decoding image with JS or Wasm should be fine. It's not video.
Video is not lossless, a second of high quality video might be a total of 5 megabytes.

A mammogram on the other hand, might be 300 images of lossless 4k resolution, which could clock in at about 2 gigabytes. That could be per breast in a given study, and a study could have prior mammograms attached as well.

You will hit memory limits, so you need to be able to unload and load data intelligently and quickly.

Image compression for mammograms seems like a thing ideally suited to a straightforward ML task — a model trained to classify sections of an image which are outside of the area of relevance (i.e. not the breast) and classify details of high importance so that detail can be retained where it's needed.

Especially handy that it wouldn't require a new file format. It only requires the encoder to support variable compression. (I know Photoshop supported this for JPEG many decades ago.)

I might be missing the details here, but everything outside the breast should be pretty much black and will get compressed very efficiently (high signal, low noise, easy to predict).

Foveated compression sounds like a super cool idea I've never thought about before.

Before that, you would need to get radiologists onboard with the idea of something being "diagnostically lossless" (think visually lossless), which is currently a hard sell (some promising research does exist on this)!

https://giannirosato.com/blog/post/nvenc-v-qsv/ This site has WASM-decoded JXL images using JXL.js. Native support would be much faster, & trying to decode much larger images with JXL.js doesn’t work very well.
What do you mean by "much faster"? Those images are opened pretty much instantly both in desktop chrome and in iOS chrome. If you didn't mention that they're JXL, I would never notice that.
Images are opened and used in a lot of places that aren't browsers.