[somishere]

Not exactly the same, but I did some work a while back retrofitting 8bit pngs as DEMs with non-linear elevation profiles (in the days when Mapbox only supported 8bit dem uploads). This allowed for fine detail in the range I was most interested in and coarser detail elsewhere. I was also working below sea level, so the standard models weren't suitable. I used gdal and node for the encode to PNG (from high res geotiffs) and then leant on mapbox expressions for the custom decode on the front end. Looked cool and file size was reasonable. Tho I'm certain much cooler things are now possible with 16 bit encode and the new terrain API.

Edit: a link to a mapbox-gl-js discussion on the use of custom dems/encodings (after which anything is possible): https://github.com/mapbox/mapbox-gl-js/issues/10775