[ttul]

SDXL actually takes conditioning from either the old or the new CLIP, or both. The malleability of SDXL is not just down to the choice of the new CLIP; the UNet itself is more opinionated.