Not exactly. They mention starting from the VAE from Stable Diffusion XL and the Transformer from Phi3.
Looks like these LLMs can really be used for anything