Hacker News new | ask | show | jobs
by conradev 996 days ago
> Carton wraps your model with some metadata and puts it in a zip file

Why a zip file?

3 comments

In addition to the benefits mentioned in the sibling comment, zip files let you seek to and access individual files in the archive without extracting all files (vs tar files for example).

This lets us do things like fetch model metadata [1] for a large remote model, by only fetching a few tiny byte ranges instead of the whole model archive.

It also means you can include sample data (images, etc) with your model and they're only fetched when necessary (for example with stable diffusion: https://carton.pub/stabilityai/sdxl)

[1] https://carton.run/docs/metadata

zip-file-as-a-container-format seems pragmatic: it's a way to bundle multiple files into one file (easier to manage than scattering multiple files), it avoids introducing a new proprietary format, it can optionally be compressed, support for reading and writing the container format is already widespread.

To give two examples of prior art, it worked for Quake 3 data files (.pk3) & geospatial data files (.kmz)

Maybe it's not the best choice but it doesn't seem like a bad one.

Also docx as well I believe.
It's a fairly common way of bundling multiple files into one that has large support and usually "good enough" compression.

It's hardly revolutionary to do this, here are some common examples of things that are zip files but don't label themselves as such:

- .jar

- .odt, .ods, .odp, .docx, .xlsx, .pptx

- .epub

- .apk

- .crx, .xpi