isn't the difference between L1, L2 and L3 also because of the functionality, not just the size+speed? L1 is data and code. L2 is data only, per core. L3 is data, synchronized between cores.
Yeah, technically there are 2 L1 caches; x86 is a 'Modified Harvard' architecture. The instruction cache also typically has to deal with caching micro-ops. I believe L2 and beyond store both instructions and data. There's also cache associativity, where the the same location in memory can be stores in one of N locations, which can differ per level. I think L1 caches are typically more associative because that takes extra silicon per byte. It looks like Zen 2 at least has an 8 way associative L1 cache.
With smaller caches and especially L1 you typically want higher associativity.
With 32KB L1 at 64B line size, you can only cache 512 lines. Grouping them in larger buckets means less spillage as hot lines randomly end up in the same bucket.