It matters if studies like this matter, that is, it matters to people who are interested in what has currently happened rather than what might happen in the future. 6 months of LLM progress keeps not looking like what I expected.
On the other hand, if you're content with your pre-existing predictions about what would happen, which I think is actually a reasonable position, there's no reason to read the paper.
That makes sense. Progress does seem very lumpy. Commercialization and secrets means that what we see publicly may not be state of the art. And I certainly never thought that Zuckerberg would be the one providing open model weights. It's a strange new world.
On the other hand, if you're content with your pre-existing predictions about what would happen, which I think is actually a reasonable position, there's no reason to read the paper.