It's fantastic that you've reduced the startup and memory overhead to the point where it's almost negligible. That's quite an achievement!
One thing that was not discussed is the impact hypervisor-based virtualization has on runtime. I've seen plenty of benchmarks where AWS EC2 instances perform much more poorly than a bare-metal machine with a similar processor. Do you have any idea what the overhead might be for clear containers vs standard linux namespace-based containers?
Thanks for sharing! So roughly 20% slower for computationally intensive workloads, likely due to nested paging putting increased pressure on the TLB. For applications using huge pages, the slowdown would likely be much less. Both docker and KVM introduce a lot of overhead with frequent, small IOs. That's likely the chattiness of the syscalls with the Kernel, which is a problem even without virtualization. Doing more work per syscall reduces those overheads. e.g. writev, readv, sendmmsg recvmmsg, etc. The context switch involved in syscalls (especially the cache and TLB pollution they cause) is very expensive.
One thing that was not discussed is the impact hypervisor-based virtualization has on runtime. I've seen plenty of benchmarks where AWS EC2 instances perform much more poorly than a bare-metal machine with a similar processor. Do you have any idea what the overhead might be for clear containers vs standard linux namespace-based containers?