Just what we need a Linux distro who's main goal is apparently to promote Intel products. The language used to describe it makes this quite clear. "The goal of Clear Linux OS, is to showcase the best of Intel Architecture technology...". This is a blatent attempt to exclude ARM who is gaining Linux market share. Whatever innovation they might bring to the table, I will avoid it purely on the basis that its aim is to benefit Intel rather than the user. Dot org my ass.
I don't think they really expect you to want to use it directly. As it says, it's a showcase. But a lot of the technology might make it into other distros in more generic forms.
One issue where "pure" containers have an advantage over VMs is IO.
For network intensive workloads, there is a choice between the efficiency of SR-IOV and the control & manageability of a virtual NIC like virtio-net. In order to get efficiency, you need to use SR-IOV, which (the last time I checked) still made lots of admins nervous when running untrusted guests. Sure, the guest could be isolated from internal resources via a vlan, but it could still be launching malicious code onto the internet, and it may be difficult to track its traffic for billing purposes, especially if you want to differentiate between external & internal traffic. SR-IOV NICs also have limited number of queues and VFs, so it is hard to over-commit servers. So in order to maintain control of guests, you end up doubling the kernel overhead by using a virtual NIC (eg, virtio-net) in the VM and a physical NIC in the hypervisor. Now you have twice the overhead, twice the packet pushing, more memory copies, VM exits, etc.
The nice thing about containers is that there is no need to choose. You get the efficiency of running just a single kernel, along with all the accounting and firewalling rules to maintain control & be able to bill the guest.
Container networking has overheads, the virtual network pairs and the natting is not costless at all, and most people with network intensive applications are allocating physical interfaces to containers anyway.
Is it bad to send an LWN Subscriber link to a large number of people? Will they get upset? They mention that they'll remove the feature if it gets abused.
"Where is it appropriate to post a subscriber link?
Almost anywhere. Private mail, messages to project mailing lists, and blog entries are all appropriate. As long as people do not use subscriber links as a way to defeat our attempts to gain subscribers, we are happy to see them shared."
How they purport to do packaging is interesting, but I'm not sure it will work well in the end. Having "bundles" that contain immutable sets of packages sounds good from a stability point of view, but unless they are entirely self contained, you'll undoubtedly run into a library that you need to updated for one bundle that then forces you to update another entire bundle. If each bundle is entirely self contained (allowing it to have it's own set of libraries), you're essentially recreating what's a static binary through package semantics. This comes with the usual downsides of static binaries.
I'm interested in seeing it tried though. The learning is in the doing.
Self contained packages are not a new idea. For example, PC-BSD has been doing this for years, via their PBI package format. See the description of PBI here: http://www.pcbsd.org/en/package-management
I think PBI does de-duplication at the package manager level by manipulating hard-links to common files, rather than installing multiple copies.
> I think PBI does de-duplication at the package manager level by manipulating hard-links to common files, rather than installing multiple copies.
Which is, itself, a bad reinvention of Plan 9's Venti filesystem. Having one, or two, or a million files on disk containing the same data should take up as much space as having just one. "Hard links" are a policy-level way to express shared mutability; deduplication of backing storage, meanwhile, should be a mechanism-level implementation detail.
ZFS has support for block-level deduplication and it comes with heavy memory and performance requirements. File-level deduplication with hard links is lightweight and requires no special support (besides a filesystem which supports hard links, obviously).
Self contained packages are not the problem, individualized libraries shipped along with those packages are.
How does PBI handle minor library version differences then? If one package provides and uses mylib-1.3.1 and another provides and uses mylib-1.3.5, how is that distinguished as the core library level (the plain .so file)? My understanding of what Clear Linux is attempting allows this level of granularity to ensure a package (really an amalgamation of individual packages in the sense of most current unixy distros) is functional and updated as a whole.
I believe that PBI works the same way. It allows different packages to independently use multiple versions of the same lib simultaneously. There is deduplication via hardlinks ONLY when the checksums match.
That's what made it attractive to me: I've painted myself into a corner several times when trying to install Ubuntu PPAs that want conflicting versions of shared libs
I'm almost of the mindset that just having userspace apps as lxc containers with everything an app needs may be better for the most-used applications...
Given how relatively cheap even fairly big SSDs are, is it really worth the storage savings for your browser to share a couple .so files with your mp3 player?
I actually like how PC-BSD pbi packages work... given the number of times solutions have been made to work around the issue and reduce space... I'm not sure it's always worth it. At least not in the desktop space.
Imagine the problem of tracking down all the different versions of a library when an exploit is found. If you have 20, or even 50 different apps that bundled openssl, imagine the hassle of making sure each one was vetted and updated as needed, not to mention the delay in getting all the different packages rebuilt and pushed (which may be a small delay, or may not, depending on the vendor).
If they are micro-vms, container-style, I don't think they will have such need to share any library? -in theory, at least- ..
I mean, it is possible to completely isolate them, all.
It may end-up very heavy though, but, and I can be wrong on this, with the constant growth of storage capacities, network bandwidth, RAM capacity, and the progress made to lighten "containers", I don't think this "heavy" downside I see of immutable infrastructures will be a real issue in the future.
No, but identifying which of your 20 micro-VMs is susceptible to the next OpenSSL exploit, and rolling out the fixes may be. It's both simpler in some aspects and more complex in others to lave local library versions for every app/service. Managing service prerequisites becomes easy and managing service feature updates becomes easier than it was, but managing service security updates becomes more complex. Juggling these different needs and capabilities is where it gets interesting.
I guess it just lead to a turning point, where end-users won't have to worry about security updates for x or y library, but more about updating the application they're using.
In the case you use containers/micro-vms, if there is a security update to do, the container "maintainer" would be in charge to push the security update, then you just need to update your container.
I'm not sure which one is the most constraining, dealing with conflicts or being careful on relating on well maintained "containers".
I guess, for production environments, the second option looks like a wise choice.
While namespacing allows you to do various things, most container setups are vm-replacements, running either full OS or not, they're used for the same purpose in the end (ie resources separation)
> No they are not. A VM is a completely different system, while a container is a packaged application.
I think you have a misunderstanding of terms here, possibly confused by all the fuzz around Docker.
A container is nothing more then a virtualization technology on OS level[1]. What you are talking about is something like rkt, which is how to run an app inside a container[2].
From the point of your app there is no difference between a VM, OpenVZ or LXC.
> What you are talking about is something like rkt, which is how to run an app inside a container
Rocket, and Docker, yes.
> I think you have a misunderstanding of terms here, possibly confused by all the fuzz around Docker
I agree Docker has spread a lot of FUD, causing great confusion about what Docker can do, but also what containers are.
> A container is nothing more then a virtualization technology on OS level[
Not quite. A container was intended to be the first (for linux) truly portable application. You create an application, "containerize" it, then you can run that application on any system with minimal effort (Ubuntu app running on CentOS, etc).
Containers are not virtualizing anything, and that is the entire point. They remove the virtualization/emulation overhead of a hypervisor and instead run your application at native speed on the native system.
Docker has tried to make a do-all application which then provides process isolation and other things to add "Security", but at the end of the day, an app running in a container on your system can still negatively impact other containers and/or the host OS (if your container needs to read/write to /etc for example).
In a VM, everything is isolated because it's literally it's own OS running on (what it thinks is) it's own hardware. An app can destroy the VM, or the VM can be exploited, but nothing outside the VM can be effected.
Xen/KVM have zero comparison to things like Rocket, and Docker.
quote: "With kvmtool, we no longer need a BIOS or UEFI; instead we can jump directly into the Linux kernel. Kvmtool is not cost-free, of course; starting kvmtool and creating the CPU contexts takes approximately 30 milliseconds."
Seeing as it's Intel I might guess that they either have extra instructions in the instruction set they know about or other optimizations that they know to look for. Seeing as they only support 4th generation and E5 v3... it wouldn't surprise me.
I dont quite understand what this is: Is it a linux distribution that can have a graphical interface like Gnome 3? My question is essentially: Is it more like Ubuntu or more like Docker?
Would very much like to see a comparison of Clear Containers and LXD. Would also like to know why Intel decided to do their own thing and not just help with the LXD project.
This does use containers and in fact they have some interesting modifications that they have made to the rkt container runtime to use KVM isolation instead of just namespaces and cgroups. See a link in 4ad's comment for an LWN article.
Those modifications are exciting for me as one of the developers of rkt. We built rkt with this concept of "stages"[1] where the rkt stage1 here is being swapped out from the default which uses "Linux containers" and instead executing lkvm. In this case the Clear Containers team was able to swap out the stage1 with some fairly minimal code changes to rkt which are going upstream. Cool stuff!