[unsnap_biceps]

Kata containers is a nice compromise. Each container is run as a microvm

[rickydroll]

On the surface, Kata appears to be a variation of LXC/LXD.

[unsnap_biceps]

LXC/LXD still shares the same host kernel. Kata runs a full instance of qemu including its own kernel.

FWIW, here's an slightly redacted example qemu instance launched by kata

/opt/kata-3.18.0/bin/qemu-system-x86_64 -name sandbox-FOO -uuid FOO -machine q35,accel=kvm,nvdimm=on -cpu host,pmu=off -qmp unix:fd=3,server=on,wait=off -m 2048M,slots=10,maxmem=1032958M -device pci-bridge,bus=pcie.0,id=pci-bridge-0,chassis_nr=1,shpc=off,addr=2,io-reserve=4k,mem-reserve=1m,pref64-reserve=1m -device virtio-serial-pci,disable-modern=false,id=serial0 -device virtconsole,chardev=charconsole0,id=console0 -chardev socket,id=charconsole0,path=/run/vc/vm/FOO/console.sock,server=on,wait=off -device nvdimm,id=nv0,memdev=mem0,unarmed=on -object memory-backend-file,id=mem0,mem-path=/opt/kata-3.18.0/share/kata-containers/kata-ubuntu-noble.image,size=268435456,readonly=on -device virtio-scsi-pci,id=scsi0,disable-modern=false -object rng-random,id=rng0,filename=/dev/urandom -device virtio-rng-pci,rng=rng0 -device vhost-vsock-pci,disable-modern=false,vhostfd=4,id=vsock-FOO,guest-cid=FOO -chardev socket,id=char-FOO,path=/run/vc/vm/FOO/vhost-fs.sock -device vhost-user-fs-pci,chardev=char-FOO,tag=kataShared,queue-size=1024 -rtc base=utc,driftfix=slew,clock=host -global kvm-pit.lost_tick_policy=discard -vga none -no-user-config -nodefaults -nographic --no-reboot -object memory-backend-file,id=dimm1,size=2048M,mem-path=/dev/shm,share=on -numa node,memdev=dimm1 -kernel /opt/kata-3.18.0/share/kata-containers/vmlinux-6.12.28-157 -append tsc=reliable no_timer_check rcupdate.rcu_expedited=1 i8042.direct=1 i8042.dumbkbd=1 i8042.nopnp=1 i8042.noaux=1 noreplace-smp reboot=k cryptomgr.notests net.ifnames=0 pci=lastbus=0 root=/dev/pmem0p1 rootflags=dax,data=ordered,errors=remount-ro ro rootfstype=ext4 console=hvc0 console=hvc1 quiet systemd.show_status=false panic=1 nr_cpus=88 selinux=0 systemd.unit=kata-containers.target systemd.mask=systemd-networkd.service systemd.mask=systemd-networkd.socket scsi_mod.scan=none cgroup_no_v1=all systemd.unified_cgroup_hierarchy=1 -pidfile /run/vc/vm/FOO/pid -smp 1,cores=1,threads=1,sockets=88,maxcpus=88

[coderatlarge]

meaning not a full-blown vm security boundary?

[unsnap_biceps]

It is

[coderatlarge]

thank you for posting the qemu details above

[coderatlarge]

thanks! so they achieve the convenience of docker with the added security of full-blown kvm? trading some perf and resource-use?

https://katacontainers.io/

[unsnap_biceps]

Yes. Microvms are stripped down to the basic hardware needed (AWS' Firecracker for example), so they 'boot' really fast, in the tenths of seconds for my containers, but you do have the extra resource overhead of running a second kernel and the performance reduction of the VM context switches. That said, it's minor enough that I feel the security tradeoff is well worth it.