Similar experience. I bought an ASUS router and enabled IPv6. It slowed down everything down. Immediately flashed OpenWrt on it, IPv6 works like charm.
It's usually bad configuration done by the router vendors. It doesn't mean IPv6 is bad.
There are maybe many buggy routers still out there that reset the IPv6 flow label field when they shouldn't, breaking hash-based load-balancers (the symptom is TCP connections spontaneously reset).
IIRC, a workaround was to prevent Linux from setting this field, or force-reset it on every outbound packet using netfilter.