[burke]

It's very heavyweight, and there's no good shared filesystem option.

We did use virtualization for a bunch of stuff before the move to Apple Silicon, back when Hypervisor.framework and xhyve actually existed and were plausibly useful.

Those also fell by the wayside in the architecture migration and now virtualization has a massive performance cost.

Apparently the M4 chips are on ARMv9 which is apparently much better at virtualization, but it remains to be seen whether apple provides anything lightweight again.

[plorkyeran]

Filesystem perf is definitely an issue, but cpu wise virtualization perf is basically free on Apple Silicon. I don’t know why you think Hypervisor.framework went away or became useless in the architecture migration. Obviously x86 VMs are slow, but we’ve been using arm64 VMs for years now with great results.

[burke]

Ok, probably I'm over-interpreting some first-year pain with the new chips.

Anyway, even at that, virtualization isn't really my preferred way to solve any of the problems I have.

[darklion]

Uh, Hypervisor still exists, and is still supported: https://developer.apple.com/documentation/hypervisor

[burke]

Yep. However, before the Apple Silicon migration, VT-x gave us extremely low-overhead virtualization. We built a tiny linux kernel that booted in a second or two and were able to run whatever we wanted with minimal perf overhead.

In the Apple Silicon migration, obviously emulating x86_64 got slow, but even when we built ARM64 VMs, performance was still miserable: there was (is?) no way -- at least no way we ever figured out -- to get reasonable perf out of virtualization on a macbook.

It's possible that this changed post-M1 and it sounds likely it's set to change with M4.

EDIT: ok, I'm probably hallucinating more problem than there actually turned out to be based on the pain in the first year of the M1 chips.

[inkyoto]

If you are referring to the nested virtualisation support in ARM v8, it was added in the ARM v8.3-A revision of the architecture, and M1 uses ARM v8.5-A as the baseline.

But yes, virtualisaiton support for ARM (in general) was abysmal and Apple Silicon was the catalyst that pushed people over the edge towards improving it across aarch64 (also in general).

[hi-v-rocknroll]

> It's very heavyweight

I guess your dainty, utopian senses are irrationally offended by something that works. Some types of virtualization offer hard isolation guarantees while cgroups, chroot, jails and the like provide pretend isolation lacking hard guarantees about either security or resource limits. KVM is tiny and so is Virtualization.framework. If you want perfect "containers", you're not going to find them anywhere because they try solve a problem (convenience, speed, and isolation) at the wrong level, in the wrong way. Type 1 Xen and VMware are the gold standards supporting all sorts of deduplication, replication, and migration options that containers can't touch. Type 2 Kata Containers is another option out there with stronger guarantees with the same interface as CRI. If these don't work for you, write a better solution that can divvy disk IOPS and latency, process manipulation, memory shares and bandwidth, network bandwidth and priority, and VFS fairly while sandboxing misbehaving processes from taking down other containers on the same host. I submit that these are essentially impossible goals with the architecture of Linux, which is why variants of virtualization providing paravirtualized guests is generally superior in providing service guarantees because there is out-of-band management exterior to fallible, DoSable containers.

[burke]

> I guess your dainty, utopian senses are irrationally offended by something that works

1) What

[talldayo]

> If these don't work for you, write a better solution

It's MacOS, you fucking can't.

[Vilian]

Bro read one line and went crazy