I think it is a bit more complicated. The rule, together with the aliasing rule, if taken a face value, means you could do unrestricted aliasing as long as you cast to an union type on access. I believe that's the interpretation the GCC Devs reject as is makes TBAA ineffective.
Instead they interpret it narrowly to only allow punning through objects that are actual unions (as described in the GCC docs).