[deadmutex]

> AVX512 isn't available on most new CPUs

Please define new. Also, I think AMD uses very similar cores in server and client. So, disabling AVX512 may be an Intel thing (my guess is that so they can easily move threads between E & P cores).

[murderfs]

They didn't disable it at first on their client CPUs, and it resulted in code randomly crashing depending on whether ifunc resolvers first ran on a big core or a little core.

It's pretty surprising that multiple CPU vendors have run into issues like this (some more than once, fucking Samsung), when it's pretty much the first thing that anyone on the toolchain side of thing asks when they hear about heterogenous cores on a CPU.

[menaerus]

Would it not be possible to solve the problem of "thread migration to an incompatible core" on the kernel scheduler level?

[adrian_b]

It would be possible, but Microsoft has already accumulated many decades of being extremely reluctant to make even the slightest changes in Windows in order to accommodate new CPU features.

Since 1990 and until now, the Intel-AMD ISA and the hardware architecture of the Intel-AMD CPUs have accumulated a large number of very undesirable features, which increase the cost of hardware and reduce the security of computers, for which the only justification was the fact that Microsoft refused to update Windows so they expected that the hardware vendors must do whatever it takes to keep their CPUs compatible with the legacy operating systems.

These workarounds have moved inside the CPUs many functions that properly belong to the operating system, like remote management and voltage-frequency control.

[menaerus]

And do you know about Linux? After cursory search I couldn't find anything as well. In this case it seems very expensive, if not impossible without the huge penalty, to implement such a workaround in the hardware.

Heterogeneous compute aware kernel scheduler seems like the most natural approach to me.

[murderfs]

The kernel can't do anything here, unless your answer is "threads can't ever migrate between types of cores, so cross your fingers on which core you spawn on". The kernel doesn't know what flavor of ISA userspace is expecting, because userspace doesn't know what flavor it wants until it actually gets to run: symbol multiversioning lets you compile a function for multiple versions of an instruction set and pick the best one at runtime.

You could imagine the kernel being clever and handling the first SIGILL in hardware and then marking the process as big-core-only, but you can't handle this generally since you can have new instructions get encoded as nop (I believe the AArch64 PAC instructions do that?). You could also have an ifunc resolver intentionally call the instruction and expect and handle the signal.

The solution is to make your hardware actually work, and not claim to support instructions that do not actually exist on all of your cores.