[jeffbee]

At least on Intel, AVX has its own functional units and register file, so I would say it's not a major concern. It's possible that running some AVX instructions could be almost or completely free if you weren't using execution port 5 anyway, to the extent that instruction-level parallelism can be considered "free".

If you're really taking a microscope to performance, the main hazards would be intermittently using AVX for only a few instructions, because that might lead to the CPU stopping for a few microseconds to turn the power on and off on the functional units. If you're using them heavily the overall thermal situation might cause a core or package-wide clock rate degradation, but if you have a use case for sustained AVX-512 usage this is likely to be a good tradeoff.