[wenc]

Not quite.

Embeddings are a form of latent variables.

Attention query/key/value vectors are latent variables.

More generally, a latent variable is any internal, not-directly-observed representation that compresses or restructures information from inputs into a form useful for producing outputs.

They usually capture some underlying behavior in either lower dimensional or otherwise compressed space.