It's interesting that Meta is heavily using Google's models (as opposed to Anthropic or OpenAI) given that they are not SOTA for coding. I wonder if this for some strategic/competitive reason, or maybe for cost saving?
I would imagine there are many situations within Meta's applications where relatively small models can do a good job — sentiment analysis, abusive language detection, characterising users based on their posts, summarising a user's complaint so it can be ignored more efficiently, assessing whether ads are likely to be fraudulent so they can be run more often, etc.
Hmm ... I was assuming they were using these models for development, but I wonder if any of it might be for production instead - perhaps using vision models to analyze posted content? That would certainly be massive scale, but I'd have thought that scale would require them to be running in their own datacenters.
OTOH, if they are stressing Google's capacity then it seems it has to be for production use, which would relfect a massive failure on Meta's side given their investment in datacenters and AI. If they can't utilize their own models and datacenters, then maybe they should just rent the excess capacity to Google! :)