[marmarama]

Unikernels never really removed the layer between the app and the kernel, they just made the hypervisor the kernel and invented a layer to handle IO to/from the virtual devices presented by the hypervisor, inside the same memory space as the app.

If the hypervisor is KVM, which they are if running on modern AWS EC2 instances or GCP, unikernel apps are literally just Linux processes; the underlying Linux host is doing all the heavy lifting. Conceptually, they're essentially the same as a sandboxed ordinary Linux process with an in-process IO stack, but without the ability to monitor or debug them as if they were an ordinary Linux process.

[eyberg]

I agree with most of this except for the monitoring/debugging part.

GDB works great as does printf and others: https://docs.ops.city/ops/debugging

Prometheus and other open source monitoring solutions work out of the box and we even have a custom APM service that is unikernel-tuned https://nanovms.com/radar .

[pjmlp]

If they are running on Hyper-V on Azure, there is no underlying kernel doing anything.

It is a matter of who is offering what, not what unikernels are capable of.

[zamadatix]

Even though Hyper-V is also a type-1 hypervisor in terms of CPU execution something still needs to mediate the virtual devices to the physical hardware and that's done by the hypervisor's kernel. In Hyper-V's case that is NT which mediates the vNIC with the virtual switch and physical NIC "uplink".

Some devices they can also support hardware assisted virtualization like for PCIe devices (NICs/NVMe storage/GPUs) via SR-IOV but it's been pretty rare to see that in practice with unikernels as they typically have limited physical device driver support on top of that not really being an option everywhere all the time as it places limitations on the cloud provider that paravirtual devices don't.

[pjmlp]

Windows is also a guest OS on Hyper-V, running on privileged guest know as root partition.

https://docs.microsoft.com/en-us/virtualization/hyper-v-on-w...

[vlovich123]

Can you explain this? Afaict hyper-v is the same as VMware or virtual box where you have a host OS and multiple guest OSes (which makes sense because you still need something to run the OS drivers). It sounds like what you’re implying is it behaves differently but I’m not sure how. Can you elaborate?

[bpye]

Hyper-V is similar to Xen. The hypervisor runs on bare metal, the Windows root is akin to your dom0 on Xen, and your guests are your domU.

[vlovich123]

Ok, but then I still don’t understand this piece:

> If they are running on Hyper-V on Azure, there is no underlying kernel doing anything.

In a Xen model as I understand it, the dom0 kernel is still actually responsible for talking to all the hardware directly and presenting a virtualized implementation that Xen can mux other guests on, no? So there’s still a kernel there and it’s doing quite a bit of work, no?

[bpye]

The dom0 kernel is responsible for talking to devices such as a disk or a NIC, yes, be that Linux, NT, *BSD, etc. The hypervisor is typically responsible for enforcing isolation of compute, memory, etc. The line does blur - for example Hyper-V may be configured with something called the “root scheduler” with which NT is responsible for scheduling virtual CPUs [0] - though the hypervisor is still enforcing the isolation.

[0] - https://docs.microsoft.com/en-us/windows-server/virtualizati...

[pjmlp]

Windows runs as guest OS on top of Hyper-V as well, it is a type 1 hypervisor.

Basically when you activate Hyper-V, you will be getting one VM running where the host is only a guest with special privileges known as root partition.

[zamadatix]

App --syscall --> kernel --hypercall--> kernel

App --"syscall"--> gVisor --syscall--> kernel

Unikernel --hypercall--> kernel

Though ideally something like SR-IOV would come into play and they hypervisor is just scheduling shared compute. Of course there is theory and reality and reality is unikernels never really caught on for many reasons while the normal stack just got optimized enough.