[stevvooe]

There is no real advantage to a dependency graph versus a list of layers. Originally, Docker images were actually fetched in sequence, similar to the ACI approach, but this blocked parallelization on each subsequent step. What has become OCI uses a list of layers to avoid this bottleneck.

As long as images share layers, network traffic would be the same in either scenario.

Reduced transfers will be achieved by formats that have a more granular filesystem representation. Right now, we are really held back by the continued use of tar, but that is another problem.

[philips]

Yes, I agree that the method that OCI v1.0 & Docker v2.2 has of telling you ahead of time "you must fetch all of these objects and they will be this size" in a compact single metadata only format is the best option. Then it is up to the fetching code to choose the best way to pipeline or parallelize.

[clhodapp]

I think that lack of parallelism has almost nothing to do with graphs vs lists. It would be pretty easy to either store the entire dependency graph's fetching-critical metadata in each image or to make it possible to fetch the metadata separately from the core data. Both of those would allow you to parallelize downloading of image data in the graph-based system.

Having more granular layers can help a bit but in practice it hits its limits very quickly because higher layers will necessarily get swept up and have to be rebuilt when layers below them that they don't actually depend on get rebuilt. This rules out having a core image and a set of modules that each downstream image may or may not need (to be mixed in or not per-image). Thus, when you use the layer-based system in practice you usually end up with very little layer sharing between images, outside of the core Linux distro layers.

[stevvooe]

Note that I did not say "more granular layers" but more "more granular filesystem representation". This is a key point in achieving better data sharing. In fact, these use cases are not ruled out, but bolstered by OCIs approach. The introduction of alternative systems will be much easier to do when OCI is broadly implemented.

As far as metadata is concerned, your suggestion is exactly what OCI does. The problem with ACI is that it embeds a large part of the metadata into a tar file, which has to be fetched in its entirety. OCI is mostly metadata scaffolding, made up of indexes, manifests and configs that can all be fetched without large bandwidth requirements.

The compositional aspect that you've brought up has been explored and it doesn't make a whole lot of sense to cram that into images. Typically, such a system requires composing container filesystems through named references, allowing components to be independently rebuilt. Because this composition often relies on details of the target deployment media (orchestration system, container runtime, specific operating system, etc.), especially in how it deals with the security of name resolution, baking it into an image format leads to massive inflexibility.

Moving it up the stack works much better and avoid the technical complexities that come with doing it within images. We can see this image composition in action in k8s PODs, docker stacks and other systems. Such constructions can be distributed through OCI images, through media types, but they are based on the compositional capabilities of the target system.