AVX code is known to make the CPU run way hotter than usual. Perhaps that caused throttling that made general code running at the same time, or within a short span thereafter, perform worse?
Note that with pstate /proc/cpuinfo is not reflective (not suggested here but, in the past, the MHz used to change to reflect the scaling speed). You could also look at 'powertop'
One way I've found to isolate this is to turn off turbo boost and underclock the cpu, while making sure the avx offset is set to 100%. While you would never want to run a production system like this, it does help to eliminate any issues with cpu throttling. If the avx-512 version of the program still runs slower then something else is interfering besides cpu throttling.