[twobitshifter]

AIExplained on youtube has guessed that Gemini 1.5 pro is taking Mistral’s accurate long content retrieval and Google just scaled it as much as they could. The Gemini 1.5 pro paper has a citation back to the last mistral paper in 2024.

[MasterScrat]

And how does Mistral do "accurate long content retrieval"?

[twobitshifter]

see the long range performance piece here https://arxiv.org/pdf/2401.04088.pdf

[Johnyjohnson123]

I don't think they explain it in the paper, do they? They just mention the result it seems. I am really curious to know too. Maybe they imply it's the result of their Mixture Of Experts architecture? Or maybe they just don't wanna say, idk