[tptacek]

That's a valid way to look at it, but there are other ways. Containers are also a simple, practical way to bundle applications and their dependencies in a relatively standardized way, so they can be run on different compute fabrics.

That sense of the term isn't loaded with any specific notion of how attack surfaces should work. I think modern "Docker"'s security properties are underrated†. But you still can't run multitenant workloads from arbitrary untrusted tenants on shared-kernel isolation. It turns out to be pretty damned useful to be able to ship application components as containers, but have them run as VMs.

† https://fly.io/blog/sandboxing-and-workload-isolation/

[Saint_Genet]

Your first paragraph pretty much sums up what docker is, it’s a convenient way to design and build a system, but it is not a security mechanism.

If you’re building a system that’s handling classified information, there is probably not an accreditation authority in the world that would let you use containers or even hypervisors as a way to separate different information classes.

[foobar33333]

Docker _should_ be secure, any part that isn't secure is a bug which can be reported. That disconnected to the reality of whether docker actually is secure, but in theory it is meant to be.

Other implementation like podman get even better security by not running as root.

[tptacek]

The fundamental flaw of the Docker container security model is the shared kernel, which is a gigantic attack surface in which vulnerabilities are present, somewhat routinely, in functionality that can't be masked off with system call filters.

The win of virtualization is that the machinery required to hypervise a kernel is much, much smaller than the kernel itself; to use the 70s terminology, it's a minimized trusted computing base.

[Saint_Genet]

Absolutely it should be as secure as possible, but the fundamental concept of what a container is means it cannot be used for some high security concepts. One of the cornerstones of classified information security is physical separation, and containers just can’t provide that.

[djmetzle]

> I think modern "Docker"'s security properties are underrated†

100% agree.

The docker/CRI-de-jour (by default) strips off many "dangerous" system capabilities. By default a pid on linux gets something like over one hundred system capabilities, and most container runtimes strip that down to around 50. Those number are not exact.

Stripping down the system level capabilities of your workload is assuredly a security improvement over running that workload "bare metal" on the system.

Ref: https://www.redhat.com/en/blog/secure-your-containers-one-we...

[globular-toast]

Technically what you're describing is an image. Might sound pedantic but interchanging container and image does often cause confusion in my experience.

[1vuio0pswjnm7]

I was making these all the time on NetBSD for dd'ing to USB sticks long before "Docker". FFS images containing only a bootloader and alternate kernels with embedded userlands. One was the "update" kernel and the other was the running kernel. I could pull out the stick after booting; I could also mount the stick and edit the update kernel. The userland was generally a single statically compiled binary, like busybox, but better. I thought about the possibilities of distributing software by bootable USB stick but not the possibilty that people might run the images in VM's.

[nijave]

>you still can't run multitenant workloads from arbitrary untrusted tenants on shared-kernel isolation

All those bargain basement OpenVZ "VPS" providers beg to differ :)

There's also gVisor

[RcouF1uZ4gsC]

> Containers are also a simple, practical way to bundle applications and their dependencies in a relatively standardized way, so they can be run on different compute fabrics.

What I find interesting, is that many uses of containers are just reinventing statically linked binaries in a more complicated form.

[nickelpro]

Those are two different things. If I need GCC 11 and its associated standard libraries and toolchain to build my app, and I need to build that app on CI services that ship an older version of GCC, I need some way to bundle my dependencies for arbitrary compute fabrics.

The answer to this is a container. Containers don't "reinvent" static linking, they solve problems that go beyond static linking.

[coldtea]

>If I need GCC 11 and its associated standard libraries and toolchain to build my app, and I need to build that app on CI services that ship an older version of GCC, I need some way to bundle my dependencies for arbitrary compute fabrics. The answer to this is a container. Containers don't "reinvent" static linking, they solve problems that go beyond static linking.

The problems they solve might go beyond "static linking", but are accidental complexity problems that doesn't go beyond namespacing.

With proper namespacing support, one could trivially build "with GCC 11 and its associated standard libraries and toolchain" on a CI service that ships with "an older version of GCC".

Ideally, one would just need a local folder with the GCC11 and its dependencies, and at most an ENV entry for where to pick up deps (optimally not even that, the GCC11 binary should give precedence to the local versions within the same folder by default).

But, no, instead we need to juggle 20 folder locations, PATH and EVN variables, burn-in build paths into libraries and executables, and so on...

[rualca]

> The answer to this is a container. Containers don't "reinvent" static linking, they solve problems that go beyond static linking.

This.

I would also point out that containers handle also other critical features like it's virtual network, which obviously goes way beyond what may be simplistically described as "static linking".

[tptacek]

They're more of a generalization of statically linked binaries than a reinvention.

[0xCMP]

I think of Docker as a universal static compiler. And I mean that in a positive way: static compiling makes a lot of sense with the incredible complexity we're often finding ourselves in. There really are no other ways to distribute node/ruby/python apps in a static way and when there is it's limited in serious ways beyond often being ONLY for a specific ecosystem (e.g. wheels for python).

[snovv_crash]

There are many ways of doing this for Python... pyinstaller, py2exe, Nuitka to name a few

[coldtea]

Python is probably the worst example for achieving this...

The fact that "there are many tools to do this for Python" is already a big red warning sign...

[RexM]

Cool, now do node and ruby, using the same tool.

[ta988]

Yes and they tend to all crumble when you have complex use cases with numpy, scipy, qt etc They can be a pain to deploy and manage remotely as well.

[flatline]

They are different means of composition. Dynamically (I assume that’s what you meant?) linked libraries allow you to compose binary artifacts. Containers allow you to compose services.

[lanstin]

Not sure what you mean. The network lets you compose services. It has for decades been possible to do that.

Containers as a form of static linking means that you ship one thing to prod and it has everything you need locally in it and it can’t be changed without you releasing a new one thing. If someone else upgrades MySQL client version on the host, your code keeps using the version you tested with, like a static binary or like a Python venv with pinned versions or vendored dependencies. It is a lot simpler to manage dependencies this way; downside is if you one of your dependencies has a security advisory, it can’t be updated by rolling out a new version by someone else. You have to update it, so unowned code becomes more expensive in that scenario.

[flatline]

Interesting. I think you are talking about intra-container composition, and I’m talking about inter-container composition. I see what you are saying (I think) about dependencies within a single container.

[brian_cunnie]

> many uses of containers are ... statically linked binaries in a more complicated form

I have found that to be true in at least one case—I had built a custom DNS server in Go (statically linked), and originally planned to run it in a container, but on further reflection realized the container brought no added value, and it was much simpler to write a systemd service control script than to bring in the extra baggage of a container ecosystem to run the DNS server.

[ric2b]

That's a very basic example. Let's say your program also depends on ffmpeg to convert some images, psql to interact with a database and a few other non-library dependencies.

With containers you can trivially ensure those are always present and with the correct versions.

Plus containers do give you some security benefits when compared to running natively.

[catlifeonmars]

Exactly! I don’t see this as a criticism of containerization so much as it is a praise of static linking.

What containerization enables is that it allows you to confer some of the advantages of static linking to languages and libraries that don’t natively support it.

[rualca]

> Exactly! I don’t see this as a criticism of containerization so much as it is a praise of static linking.

Not really. It seems the keyword "static linking" is being abused to refer to stand-alone executables, because that's what some people know. Yet, calling containers a kind of "static linking" is simplistic and incorrect, even taking the standalone executable interpretation info account.

If anything, container images are installers, and containers are the end-result of installing and configuring these containers, which is barely noticed because it works so well even and specially the networking part. More importantly, containers are designed to be both ephemeral and support multiple instances running in parallel on the same machine.

Then there's also the support for healthchecks, which allows container engines to not only determine when they should regenerate containers, but also provides out-of-the-box support for blue-green deployments.

And absolutely none of this fits the "static linking" metaphor.

[vladvasiliu]

I'd say that the static linking metaphor refers to the container image itself, in that it's standalone and (fairly) "universal".

All the other things you talk about could be set up for standalone binaries as some form of orchestration, after installation, as you say.

To me, the analogy doesn't have to be 1:1 for it to work. Yes, that means there are edge cases which should be taken into account, but that doesn't make it useless.

Especially if you look at how most developers see containers: "I'll give you this image, which I know works [in a given way] and you can run it with something that understands it". You can go ahead and set up a full K8S cluster to run it, or you can run it on Docker Desktop on a random Windows / Mac laptop. "It just works", and it's in this I think the "universal static binary" analogy works.

I get the feeling that all those fancy orchestration tools (health checks, blue / green deployment / fancy network setup / etc) have seen an impressive growth around container runtimes, and therefore often thought of as belonging together, but I don't think that one requires the other and couldn't exist without the other.

I'm curious to see what kind of ecosystem will grow around new developments such as Firecracker and "unikernel containers". I seem to remember a post on HN the other day about some effort by google to run go binaries directly on some VM kernel.

[rualca]

> I'd say that the static linking metaphor refers to the container image itself, in that it's standalone and (fairly) "universal".

That metaphor makes no sense beyond the standalone part.

Container images are way more than mere stand-alone statically linked binaries, much like an installer (deb/rpm/MSI/PKG/etc) are way more than statically linked binaries.

If anything, container images are a kin to fat JARS or macOS's bundles, but even that metaphor leaves key features out, like healthchecks, image inheritance, and of course support for software defined networking.

> To me, the analogy doesn't have to be 1:1 for it to work.

The whole point is that the metaphor makes no sense and completely misses the whole point of containers.

No one uses containers because they want statically linked binaries. Or even installers or packages. At all.

What container users want is what containers provide and neither statically linked binaries or zips or installers or bundles come close to offer. We're talking about being able to deploy and scale the same app at will in a completely sandboxed and controlled environment. We're talking about treating everything as cattle, from services to system architecture. We're talking about out-of-the-box support for healthchecks, and restart apps when they fail.

Each and every single one of these features is made available with docker run. That's what containers offer.

None of this has anything to do with static linking anything, and insisting on this metaphor shows that people completely miss the whole point of containers.

[nyx__]

Containers aren't Docker. They're a combination of Linux features. Your container runtime might provide health checks and fancy features, but containers do not.

> We're talking about being able to deploy and scale the same app at will in a completely sandboxed and controlled environment. We're talking about treating everything as cattle, from services to system architecture.

Same is true for AMIs and VMs. Containers are the technology, not the pattern.

[catlifeonmars]

In practice, I think the analogy holds up. A sizable chunk of use cases for containerization is to confer the benefits of standalone executableness, which is similar to a sizable chunk of use cases for statically linked binaries.

[rualca]

> A sizable chunk of use cases for containerization is to confer the benefits of standalone executableness,

No, not really. Just because in the end you get to run an application that does not mean that it's reasonable to explain thins in terms of static linking.

This broken metaphor is particularly counterproductive once you take into account that container images also provide you the tools to bundle interpreters, run shells, and create temporary file systems automatically.

Talking about containers in any way that leaves out the containing part is counterproductive and a bad mental model.

[catlifeonmars]

Yes those are things you can do with containers. I don’t get your point. It’s not like a said containers are only useful for bundling dependencies.

As an aside, I think we’re approaching the concept from two different perspectives. I’m working backwards from a (sub)set of use cases, while you appear to be working forwards from a set of capabilities. I think both approaches have their place.

[catlifeonmars]

> More importantly, containers are designed to be both ephemeral and support multiple instances running in parallel on the same machine.

This sounds a lot like how all os processes work (regardless of linking type of a binary).

[rualca]

You don't create persistent file system mounts by launching a process, nor do you mount files into the process's file system.

[otabdeveloper4]

A Docker image is really just a .tar.gz under the hood, with a little bit of metadata.

A Docker image is really just a chroot + some cgroups resource limits.

[amarshall]

> A Docker image is really just a chroot + some cgroups resource limits.

No, because an image specifies nothing about the runtime. Just add a Kernel and bootloader and one can boot most images. Further most container runtimes include a lot more than chroot and resource limits. Namespace isolation (process, user, network), seccomp rules, SELinux contexts, etc.

[otabdeveloper4]

> Just add a Kernel and bootloader and one can boot most images.

Certainly not true of any of the images I work with.

[amarshall]

What sort of images are you working with? I can fairly straightforwardly boot debian:stable with no modifications to the image using direct kernel boot. Is everything perfect? No, but it does boot.

[otabdeveloper4]

None of my images bundle a Linux distribution.

(Indeed, most don't even bundle bash or coreutils.)

[globular-toast]

Static linking was done out of necessity more than anything. It's the obvious way to compile a program. The fact that you got a single distributable executable was merely a convenient side effect.

[mrweasel]

If it was just a replacement for statically linked binaries, I’d be less concerned. In reality people stuff EVERYTHING in into containers, database, queue, a webserver and your application, it all goes into one container.

[crummy]

I thought one of the first rules was one container per app/service/whatever.

[dspillett]

The first rule of rules of to not expect people to follow any particular rule!

There is nothing enforcing the mapping of one container to one service so people will have multiple if they find it convenient, or sometimes if they simply don't know the "rule".

A lot of people use containers as light weight VMs. Some use them as not so light VMs, in fact. In that case multiple services in one is practically expected.

[mrweasel]

Correct, you also shouldn’t have supervisor processes in a container, as this prevents detection of crashed containers.

We’ve seen container with all sorts of weirdness, where half the service could crash and Docker would never notice, because the supervisor process was still running and that was the entry point for the image.

[tjpnz]

You might be able to get away with that for a development environment ala Vagrant. But doing that in production sounds scary, and I'm saying that as a mere dev (who has very little to do with ops).

[echelon]

> In reality people stuff EVERYTHING in into containers, database, queue, a webserver and your application, it all goes into one container.

That's simply doing it wrong! If people are doing this, you can't point to containers as the problem.