[marmarama]

https://github.com/google/gvisor gives you essentially the same benefits as a unikernel without having to compromise on compatibility or recompile your apps, and integrates nicely with Kubernetes already. It also doesn't require a hypervisor at all.

[zamadatix]

I always thought of gVisor as being in the opposite direction of the main point of unikernels in that it's another layer between the app and the kernel rather than removing the separation completely.

[marmarama]

Unikernels never really removed the layer between the app and the kernel, they just made the hypervisor the kernel and invented a layer to handle IO to/from the virtual devices presented by the hypervisor, inside the same memory space as the app.

If the hypervisor is KVM, which they are if running on modern AWS EC2 instances or GCP, unikernel apps are literally just Linux processes; the underlying Linux host is doing all the heavy lifting. Conceptually, they're essentially the same as a sandboxed ordinary Linux process with an in-process IO stack, but without the ability to monitor or debug them as if they were an ordinary Linux process.

[eyberg]

I agree with most of this except for the monitoring/debugging part.

GDB works great as does printf and others: https://docs.ops.city/ops/debugging

Prometheus and other open source monitoring solutions work out of the box and we even have a custom APM service that is unikernel-tuned https://nanovms.com/radar .

[pjmlp]

If they are running on Hyper-V on Azure, there is no underlying kernel doing anything.

It is a matter of who is offering what, not what unikernels are capable of.

[zamadatix]

Even though Hyper-V is also a type-1 hypervisor in terms of CPU execution something still needs to mediate the virtual devices to the physical hardware and that's done by the hypervisor's kernel. In Hyper-V's case that is NT which mediates the vNIC with the virtual switch and physical NIC "uplink".

Some devices they can also support hardware assisted virtualization like for PCIe devices (NICs/NVMe storage/GPUs) via SR-IOV but it's been pretty rare to see that in practice with unikernels as they typically have limited physical device driver support on top of that not really being an option everywhere all the time as it places limitations on the cloud provider that paravirtual devices don't.

[pjmlp]

Windows is also a guest OS on Hyper-V, running on privileged guest know as root partition.

https://docs.microsoft.com/en-us/virtualization/hyper-v-on-w...

[vlovich123]

Can you explain this? Afaict hyper-v is the same as VMware or virtual box where you have a host OS and multiple guest OSes (which makes sense because you still need something to run the OS drivers). It sounds like what you’re implying is it behaves differently but I’m not sure how. Can you elaborate?

[bpye]

Hyper-V is similar to Xen. The hypervisor runs on bare metal, the Windows root is akin to your dom0 on Xen, and your guests are your domU.

[vlovich123]

Ok, but then I still don’t understand this piece:

> If they are running on Hyper-V on Azure, there is no underlying kernel doing anything.

In a Xen model as I understand it, the dom0 kernel is still actually responsible for talking to all the hardware directly and presenting a virtualized implementation that Xen can mux other guests on, no? So there’s still a kernel there and it’s doing quite a bit of work, no?

[pjmlp]

Windows runs as guest OS on top of Hyper-V as well, it is a type 1 hypervisor.

Basically when you activate Hyper-V, you will be getting one VM running where the host is only a guest with special privileges known as root partition.

[zamadatix]

App --syscall --> kernel --hypercall--> kernel

App --"syscall"--> gVisor --syscall--> kernel

Unikernel --hypercall--> kernel

Though ideally something like SR-IOV would come into play and they hypervisor is just scheduling shared compute. Of course there is theory and reality and reality is unikernels never really caught on for many reasons while the normal stack just got optimized enough.

[bitcharmer]

Since this looks like an intermediary layer between userspace and the host kernel (at least if I'm reading it correctly), does anyone know what its performance impact is?

[marmarama]

The gVisor documentation has performance comparisons vs. cgroup-style 'traditional' containers at https://gvisor.dev/docs/architecture_guide/performance/

There is definitely some performance overhead, but in most cases it is less than hypervisor-based approaches.

[eyberg]

In gVisor if you aren't using hardware acceleration (eg: virtualization) then you are using ptrace which is incredibly slow.

[marmarama]

The benchmarks in the gVisor docs above are using ptrace, and they don't look too shabby.

[eyberg]

Using redis as an example it's basically half in every benchmark:

https://gvisor.dev/docs/architecture_guide/performance/#syst...

IO bound tasks can be up to 10x slower using ptrace. I think using hardware acceleration gives you acceptable performance but ptrace is just a non-starter for prod.