Linux already tracks how long ago a task used AVX-512. I assume the same mechanism could be used to track AVX as well.
https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git/...