Images are much bigger than javascript libraries. So by comparison it's a small library. And you're probably decoding, not transcoding.
You want to get browser support, but a gap-filler can keep you from waiting forever to reach 99.9%. And sometimes bandwidth use is more important than clunkiness.
JPEG XR is only supported by IE (since version 2011) and Edge before the switch to Blink/Chromium. Webp is supported by everyone except Safari and IE, but Safari and mobile Safari have about 15-20% combined market share.
New formats are basically only relevant if they get blessed by Google/Chrome, and even then you often need fallbacks for a long time.
Many browsers support some formats that are favorites of some vendor, but the new formats are not supported cross-browser, so the benefits of switching formats are reduced and the costs are raised -- in particular, one benefit of smaller files is reduced storage cost. If you have to support JPEG for legacy and then something else for Microsoft's format and something else for Google you are increasing your storage cost, encoding complexity, etc.
I looked at alternative image formats for a photo site I was working on and never convinced myself that the benefits of switching formats was worth the trouble.