[niftich]

This seems conceptually most similar to Fabrice Bellard's proposal 'BPG' [1], which was essentially a lightweight wrapper around an HEVC I-frame. The format got small amounts of attention in encoding circles, but didn't pick up mainstream traction.

The image encoding layer of HEIF is the same HEVC, but the container is MPEG-4 Part 12 (the Quicktime-descendant ISO Base Media Format; the core behind .mp4, .3gp, etc.), which theoretically gives it wide support. In this structure, HEIF supports image sequences, which will help it compete with animated GIF, GIFV (which is really just a short MPEG-4 Part 14 file containing usually an H.264 video track), animated WebP (who uses these?), and WebM.

This format doesn't really blaze new ground (EDIT: it does in the sense that it defines a container format to express image-y constructs like still images and sequences of images in an ISO media container, but see my other comment that asks how this is similar but different to video [3]), but if this repackaging and the resulting code donation and political clout helps it gain traction, we still would gain a lot.

The problem, of course, is always with backwards-compatibility. WebP was aggressively promoted by Google the same way Microsoft used to promote its quirky Windows Media formats back in the early 2000s, but the WebP and non-WebP camps are still largely separate. This is unfortunate, because WebP in my opinion really isn't very good, and a backwards-compatible compressor on top of JPEG, such as Dropbox's Lepton [2] achieves similar results. Any HEVC-based image compressor is necessarily incompatible with JPEG; so broad buy-in would be required from makers of software and hardware products, from operating systems, file managers, image viewers, image editors, cameras, and the like, to really enjoy its improved compression rates, vs. being just another incompatible format that only functions inside controlled ecosystems.

The video codec AV1 currently in development was supposed to be a grand alliance of disparate companies to agree on a common format for the future, and rectify the WebM vs. non-WebM split, but continuing to promulgate sophisticated formats based on work outside of this scope (such as MPEG- or ITU-derived custom formats) just muddies this further.

[1] https://bellard.org/bpg/ [2] https://news.ycombinator.com/item?id=13230805 [3] https://news.ycombinator.com/item?id=14490734

[Veratyr]

> The video codec AV1 currently in development was supposed to be a grand alliance of disparate companies to agree on a common format for the future, and rectify the WebM vs. non-WebM split, but continuing to promulgate sophisticated formats based on work outside of this scope (such as MPEG- or ITU-derived custom formats) just muddies this further.

Apple is just blatantly not interested in cooperating with the rest of the market. Mozilla, Google and Microsoft are all part of AoM, along with a bunch of hardware companies and distributors yet Apple is absent and apparently pursuing HEVC instead.

This isn't new either, Apple pushed HLS when others were pushing DASH and MPEG-TS when others were pushing fragmented MP4.

They seem really determined to go their own way and ignore everyone else for some reason.

[threeseed]

You're talking nonsense. H.265 is an ITU-T standard and was adopted as the standard for broadcast television ATSC. This isn't an Apple developed codec.

And many more companies support it over AoM: http://www.mpegla.com/main/programs/HEVC/Pages/Licensors.asp... http://www.mpegla.com/main/programs/HEVC/Pages/Licensees.asp... http://www.hevcadvance.com/pdfnew/LicensorList.pdf http://www.hevcadvance.com/pdfnew/LicenseeList.pdf

Including the likes of: BBC, Samsung, Dolby, GE, MediaTek, Philips, Mitsubishi, Warner Bros, Sony etc. And as for hardware vendors well AMD, Intel, Nvidia etc all support H.265.

[Veratyr]

> This isn't an Apple developed codec.

I didn't say it was, I said Apple was pursuing it.

> And many more companies support it over AoM

You can't claim that yet as AV1 isn't yet finalized.

[abritinthebay]

> You can't claim that yet as AV1 isn't yet finalized.

Do more companies support it? Yes.

Failure to ship is also a competitive disadvantage - AV1 may be great (we don't know) but if this one gets traction first that may not matter.

[lern_too_spel]

HEVC is competing against VP9. AV1 is competing against the successor to HEVC.

[zokier]

I think the significance of compatibility is increasingly diminishing due the effect of "clouds" and other walled gardens that handle your data in a more integrated manner. With the advent of wasm etc it is not completely unreasonable to ship your own image decoder with the application, and optionally a JPEG encoder for on-demand client side re-encoding for exporting images.

There is just less expectations to be able to interoperate via "dumb files" in the "app ecosystem" where the apps run in their own isolated sandboxes and import/export are explicit actions.

Not saying that this is the best thing, but the situation is different enough from the time when WebP/WMP were introduced that HEIF actually might have a fighting chance. The fact that HEIF appears to be far bigger improvement over JPEG than previous efforts of course also helps.

[TheCoreh]

Shipping a JPEG encoder is not necessary: all browsers support encoding images via toDataURL('image/jpeg') and toBlob(...)

[nomel]

I think he meant non-standard, and if not, his point still stands: with webasm, you might not have to worry about silly client side things like codecs.

[jpap]

> This format doesn't really blaze new ground (EDIT: it does in the sense that it defines a container format to express image-y constructs like still images and sequences of images in an ISO media container, but see my other comment that asks how this is similar but different to video [3]), but if this repackaging and the resulting code donation and political clout helps it gain traction, we still would gain a lot.

You need to think about HEIF as the container and disconnected from the codec used to compress the media content. In that sense, HEIF is a massive trailblazer. It gives us an ISO standard to describe image and mixed media consistent with existing methods (i.e. MP4) while allowing for new constructs (i.e. useful features of the future in addition to tiling, depth map, stereo, etc.) and new codecs as they are developed and adopted. HEIF is a universal forward-looking format.

Until now, almost all of the major imaging formats were tied to the codec and not terribly extensible to new use cases. While BPG is a great format in the short term (it gave us HEVC coded images around 2.5 years ago), it isn't an ideal choice for the long term when viewed as above.

[niutech]

TIFF and Exif (which JPEG is based on) are not tied to any particular codec and are extensible, so HEIF is not a game changer.

BTW: Did you know that you can embed LPCM/μ-Law PCM/ADPCM audio data in JPEG/Exif?

[jpap]

> TIFF and Exif (which JPEG is based on) are not tied to any particular codec and are extensible, so HEIF is not a game changer.

TIFF is a great format but its ability for extensibility is limited. You can't readily contain in a video track in addition to a photo, or a burst sequence that utilizes intra-frame prediction.

> BTW: Did you know that you can embed LPCM/μ-Law PCM/ADPCM audio data in JPEG/Exif?

Yes; I once owned point-and-shoot cameras that did this. The audio was pretty poor quality because they didn't employ compression and wanted to keep the file sizes small.

However you're limited to the 64k maximum JPEG marker segment size for non-image metadata; or ugly hacks like chaining segments (e.g. ICC profiles). Exif is strictly limited to being contained within 64k. How big is your audio track again? ;-)

JPEG has had its time as a wildly successful format but has also held back the imaging world from adopting a standardized way to cater for new applications; burst above, mixed media (photo + video + audio), depth maps, stereo images, alpha (that's not a hack), lossless, etc., etc. HEIF has all the key ingredients, including extensibility, to support these modern applications and grow as the universal container format for decades to come.

[strngtn]

That should be very high requirement, even more than video. Why not just rename it to vedio? We just need an image, not a video that called image.

[wereHamster]

At last for the web (which is what WebP targets mostly, right?) you don't need broad support. We have the new <picture> element where the UA selects the most appropriate image from a list of options. So Chrome will prefer webp, other browsers will pick jpeg, png or whatever else they support at the moment.

[niftich]

The problem is, content delivered over the web "leaks out" in standard usage and interfaces with the overworld, unless prevented by DRM. People save files, sometimes using contrived workflows, and content that was transferred to a browser for display (such as WebP) will eventually be downloaded, shared, and reposted somewhere else. Facebook once famously migrated serving most images as WebP and had to back out because people were downloading images, only to end up with .webp files that are unsupported in other programs [1].

In the past, I articulated, on the topic of 'Ask HN: Software for writing a diary that will be around in 20 years from now' [2]:

I'd be wary of the archival potential of formats that are solely used on the Web with little usage on tangible physical hardware by major commercial publishers -- the Web of today moves very fast and technologies come and go. Google is pushing WebP, WebM, but work is already under way on a big consensus format called AV1. When AV1 comes out, new VP8/VP9 content will likely no longer be produced. Browsers periodically prune older features, 20 years is almost as long as the web has been around, and given enough time support for the format may only be available in software that make format coverage an explicit goal (ffmpeg, libav, VLC). Opus is being made a mandatory audio codec for WebRTC, teleconferencing is usually ephemeral -- will there be lots of .opus files sitting on disks in the future? Too early to tell, not worth gambling on.

The context of this was choosing formats for the express purpose of long-term archival, but our incidental usage of formats today will shape the sort of files that will be naturally around in our future.

[1] https://news.ycombinator.com/item?id=5589206 [2] https://news.ycombinator.com/item?id=12979854#12980359

[ovao]

This is why many tend to consider only uncompressed/lossless formats viable for archival usage. Doubling down on those doesn't really address what you were talking about — something that you make today being viewable in browsers of the future — but with the original content itself preserved in a way that maintains its full quality, the 'deliverable' formats can be updated periodically from the original masters in a way that the content will be viewable with software 20 years from now, and without degrading quality each time.

We had similar problems in the past with physical media, and still do to some extent, but in the purely digital domain this problem is somewhat more tractable (content negotiation helps facilitate these transitions for those willing to work that into build pipelines and maintain those over time, for example.)

[jacobolus]

If possible, I’d much prefer having a single format that is widely supported, hardware accelerated, and with a state-of-the-art codec. Making content authors create multiple different files in different formats—some with deficient feature support—sounds like a pain in the butt for everyone.

If we can get everyone to agree on a format that handles layers, exposure bracketing, transparency, animation, high bit depth, different color models, both lossy and lossless compression, every common type of metadata, etc., that would be great.

[astrange]

If your format handles everything it isn't a format, it just is everything. (The one you want is TIFF though.)

[jacobolus]

The point is that you want support for those features (certainly the core codecs, etc.) in the spec, so if you send someone a file you can be sure they can read it.

TIFF is not useful if you want to use a state-of-the-art lossy codec and get hardware-accelerated decoding on an arbitrary mobile device, or if you want to display an animated image with transparency on a web page.

[astrange]

But can you actually get people to implement it if you make it that complicated? Images are pretty hard, hardware support is /actually/ really hard.

That's why it's best to accept whatever format is already out there and author to it - the only thing that matters is that it actually works when it's out there.

Actually I'm surprised there's still no good way to do animations with transparency, but maybe nobody wants to use it?

[jpap]

Yes! This is one of the motivations behind HEIF: it's codec agnostic and extensible. You can use HEIF with JPEG and AVC for example, or some new codec down the line.

It's also extensible, in that you can define new item types and box structures to describe any new imaging construct you like. For example, you could host images with alpha, depth maps, stereo left/right channels, a HDR image with each of the raw brackets and the final fused image, etc.

The format relies upon the ISO Base Media File Format file-type branding to distinguish the infinite possibilities into a clear set of capabilities required for playback; in the same way MPEG4 does for video codec profile and levels.

[nkkollaw]

There are services like Imgix that will do the conversion automatically on the fly, you don't have to convert images manually.

[zerocrates]

It mentions that it supports not only sequences but inter-frame prediction (so P- and possibly B- frames I assume), which is sort of surprising to me but of course makes sense especially given Apple's "image burst" usage. I'm also suprised to learn that WebP apparently supports P-frames in its animations.

What's really different between this and the existing usages of the MP4 container for images, though? Is it just the packaging as an "image" format, or specification of more image-y metadata?

[jpap]

HEIF is actually very closely related to MP4 (or really, the ISO Base Media File Format (BMFF); and historically, QuickTime which is what they are both based on).

You are right -- the main difference is that BMFF was originally designed for time-based media (video, audio) but isn't in itself well suited towards media that isn't time based, like images and their associated metadata (e.g. Exif).

HEIF extends the ISO BMFF so that you can have untimed media -- one or more photos, a collection -- or even mixed media comprising both untimed and timed media. Apple's live photo is a good example of mixed media, comprising a photo and a video.

But the HEIF container format goes so much further. You could have image sequences -- for example a higher-quality version of Animated GIF (using I-frames only, or P/B for more compression) with looping -- tiling, auxiliary images like depth maps and alpha channels, or even stereo images with a left and right channel.

Though originally HEIF came out of the HEVC standards track, hence the words "High Efficiency" in the name, it was later extended to include other codecs like JPEG, AVC. There's no reason it couldn't be extended to include VP9, PNG, or any other codec.

Think of HEIF as a versatile, extensible, standardized container format. The media coding scheme is separable. This is a big deal for the future of image/media coding because we're no-longer locked into "yet another format" that's tied to the codec. With the ISO BMFF box model, HEIF can grow to adapt new constructs and codecs for some generations to come.

[vetinari]

> Think of HEIF as a versatile, extensible, standardized container format. The media coding scheme is separable. This is a big deal for the future of image/media coding because we're no-longer locked into "yet another format" that's tied to the codec.

There were similar approaches in the past and they failed. Electronic Art's IFF, or later TIFF, were also designed as extensible containers. The vast majority of the software handling these formats handles only the most popular codecs, the fringe one pushing it forward will get ignored.

[jacobolus]

https://nokiatech.github.io/heif/technical.html