[nexuist]

LXD is so weird. It feels like the halfway point between chroot and Docker proper. Can someone more familiar with containers explain what its usecase is meant to be?

For reference my only interaction with it was for privilege escalation on HackTheBox: https://www.hackingarticles.in/lxd-privilege-escalation/

[dragonsh]

Not sure if you know docker was built on top of LXC container and later moved to write its own library trying to replicate lxc. In general LXD is more secure than docker in its general configuration as containers in LXD are mapped to userid and use shiftfs [1], nothing like this in Docker yet including kubernetes. Usually docker always had more security vulnerabilities than LXD.

So LXD is much better than docker except docker in spite of being an inferior solution became popular with marketing money spend on it due to hype. LXD stayed with people who believe in pragmatic simplicity. Docker is plagued by privilege escalation for a very long time. Check the details in general Docker has more vulnerabilities than LXD. [2] [3]

[1] https://lwn.net/Articles/687354/

[2] https://www.cvedetails.com/vulnerability-list/vendor_id-1353...

[3] https://www.cvedetails.com/vulnerability-list/vendor_id-1313...

[cheph]

> So LXD is much better than docker except docker in spite of being an inferior solution became popular with marketing money spend on it due to hype.

This is kind of like saying a tractor is safer than a car. LXD does not have the same feature set as docker or k8s.

If all you want is a more secure docker then podman got you covered.

If you want more secure k8s then you will have to wait a bit I guess for https://github.com/rootless-containers/usernetes but the cri-o runtime for k8s does have a rootless mode.

LXD is not a replacement for docker or for k8s as it offers a different feature set from both of those.

Last I wanted what one of the features it offered, which is a persistent whole OS container, I tried to install LXD on fedora, and after trying to get lxc running, failing to do so and seeing it's horror show of a systemd setup while debugging [1], I looked elsewhere and instead settled for rootless podman with --rootfs.

[1] https://github.com/lxc/lxc/blob/master/config/init/systemd/l... does something which ubuntu is very fond off and is the main reason why I stopped using it. They call sysv-style init scripts like https://github.com/lxc/lxc/blob/master/config/init/sysvinit/... from systemd and in my experience this obsecures errors as somethign ends up failing but via systemd it still looks like it is running.

[dragonsh]

Well LXD, Docker and k8s have a lot of overlap in functionality and features, so it’s just disingenuous to say otherwise.

For over 90% of startups Docker with k8s is just not necessary and tie them to managed version of specific vendor or cloud provider with higher costs and a lot of overheads. It’s pretty hard for a startup to manage self hosted k8s, given large number of moving parts and management of k8s infrastructure is as big as a task of managing the startup product itself.

LXD is decent enough to build a good high available horizontally scalable cluster on cloud of your choice or bare metal and can be managed by startup teams. Obviously once the startup begins to reach millions of customers and users and have enough revenue than k8s might be viable.

In majority of the projects LXD provides much better infrastructure.

[cheph]

> Well LXD, Docker and k8s have a lot of overlap in functionality and features, so it’s just disingenuous to say otherwise.

Tractors and cars have a lot of overlap also. They are still different things and people who want one normally do not consider the other as an alternative.

> For over 90% of startups Docker with k8s is just not necessary and tie them to managed version of specific vendor or cloud provider with higher costs and a lot of overheads.

This has not been my experience but at least there are managed k8s offerings from multiple vendors with some intersection of functionality. Sure different vendors have different extensions but there is only one LXD vendor and the support for LXD is kind of flaky in my experience whereas I have ran 3 different k8s distros on my fedora laptop without really breaking a sweat.

> In majority of the projects LXD provides much better infrastructure.

If you are happy with what LXD provides, and if you can actually use it, then great use it. I have yet to be in a situation where I could use it or where I really wanted what it offers. I tried installing it via snap, I tried installing it from copr, I tried manually installing it, then I concluded that I don't actually want anything from it that I cannot get elsewhere.

[martinmunk]

I run LXD on some machines at work and overall I love it. Think of it as VM hosting, but very little overhead thanks to the shared kernel. (But can for some reason now also be used to manage KVM VM's) Generally I use it for two use-cases, where I need to do minimal work:

* Giving co-workers a container they can use to host small utilities on a static IP to share, instead of them hosting it on their own desktop. Think compiler-explorer ect.

* Giving co-workers access to our 64 core ThreadRipper for heavy workloads, sandboxed from one another.

But a mandatory "F* Snap" needs to be said.

[Izkata]

> but very little overhead thanks to the shared kernel. (But can for some reason now also be used to manage KVM VM's)

Which means you can now use the same interface for, say, a Windows VM alongside your lightweight linux containers.

[paraphrased from a reddit post, I was wondering something similar a few weeks ago but haven't done it myself]

[kingosticks]

When does the sandboxing become useful between co-workers? We have a bunch of powerful workstations but we just ssh in and use them. Does this facilitate users installing packages system-wide in their sandbox? Honest question, just trying to work out what we might be missing wit our historic setup.

[martinmunk]

> Does this facilitate users installing packages system-wide in their sandbox?

This. They can use it as they please, like a second desktop. I can limit their resources, migrate or archive the containers as needed.

One guy was training some ML, one was messing with compiler versions, one with Debian packaging tools ect.

[cheph]

To me most Ubuntu things like LXD, Mir, snap feels like it was made by someone who did not understand the existing solutions and could not be bothered to understand it. There may be some points where LXD is "better" (they keep bringing up uid/gid mapping) but it really does not provide the same functionality as OCI containers and does not enable the same workflows as OCI containers.

LXD is not an alternative to Docker or K8S, it is something different which offers different features.

And if we are just talking docker, and not k8s, then all the security you can ever want can be found in podman which by default operates rootless and daemonless and works on stock standard OCI containers.

If we are talking k8s there are already runtimes which support rootless operation like cri-o and there are k8s distros that support rootless operation https://github.com/rootless-containers/usernetes - these maybe are not as widely used as they should be and work is ongoing but you will soon see more of them I think.

[dragonsh]

LXC existed before Docker and indeed as I said Docker initially built on top of LXC. So LXD is not an afterthought as you tried to put it, it’s way before Docker and k8s. Docker and k8s became popular given marketing money put on them.

Also rootless container has been a feature of LXC since 1.0 in 2013-14, which could not be incorporated in Docker as they tried to re-invent the wheels by writing their own libcontainer which eventually resulted in many vulnerabilities which even impacted k8s even in 2019.

Still today unless one use a managed version of k8s or use managed service by major cloud provider the infrastructure will be insecure with k8s given most of the Docker images still not tested as rootless containers. Also for a small team it’s pretty hard to have secure self-hosted k8s infrastructure given sheer complexity and moving parts.

[fomine3]

Previously LXC/LXD/Zones/Jail was "normal" container (similar use as virtualization) and Docker was weird technology before Docker is getting popular.

[panpanna]

Think halfway between docker and vagrant instead.

I think Mark Shuttleworth explained LXD best:

https://www.youtube.com/watch?v=0z3yusiCOCk