AV1 is more complex. Prediction can be more complex with things like "combined inter/intra" or "warped motion" or "overlapping block MC" compared to HEVC/VP9. Then there's additional postfilters like loop restoration, cdef and film grain that didn't exist in VP9 (just deblock - which also exists in AV1) or HEVC (deblock + sao). Entropy coding is more expensive than VP9 with per-symbol entropy updates (which HEVC also has). Bigger blocks are probably an overall win, but bigger transforms can be painful with many non-zero coefficients. And intermediates in 2D MC and bi-directional ("compound") prediction are 12-bit instead of 8-bit in VP9 for SDR/8bit video. This is more similar to HEVC. So overall, AV1 > HEVC > VP9 in terms of runtime complexity, this is expected, nothing you can do about it.
It looks like dav1d is (or was 2 years ago, maybe it's even faster now) on par with ffmpeg's hevc decoder in that aspect: https://youtu.be/wkZ4KfZ7x1k?t=656
You're right about VP9 though, definitely faster to decode, though there are trade-offs (video quality, encoding performance) when compared to HEVC.
I wonder if it's just because it's new. I remember years ago the first 1.x versions of libopus used way more CPU than Vorbis to decode, now they're comparable. (This was on a teeny little chip where decoding 1 Vorbis stream took a measurable amount of CPU)