Attention query/key/value vectors are latent variables.
More generally, a latent variable is any internal, not-directly-observed representation that compresses or restructures information from inputs into a form useful for producing outputs.
They usually capture some underlying behavior in either lower dimensional or otherwise compressed space.
Embeddings are a form of latent variables.
Attention query/key/value vectors are latent variables.
More generally, a latent variable is any internal, not-directly-observed representation that compresses or restructures information from inputs into a form useful for producing outputs.
They usually capture some underlying behavior in either lower dimensional or otherwise compressed space.