[lossolo]

For anyone interested it's called retrieval transformer. Here is example of one from Google: RETRO[1][2]

1. https://arxiv.org/abs/2112.04426

2. https://jalammar.github.io/illustrated-retrieval-transformer...