[ben_w]

Both, I think. However I would also add that my understanding is second-hand, reading and watching what others have to say rather than any deep academic research of my own.

Every source I've read or watched all agree the sample sizes used when creating the tests in the first place just aren't large enough to be all that confident beyond 2σ.

There's also the problem that training on tests is effective, which limits the ability of a test to be validated on those taking it fairly soon after it gets published.

A further other issue is that most IQ tests pre-suppose the existence of a G-factor and treat differences between types of intelligence as noise rather than a signal for the existence of any multi-factor variation. I don't know which hypothesis is correct (I lean towards thinking it's mostly a G-factor, but there are clearly some things which can only be explained by multi-factor models, e.g. where a skill is entirely absent such as in the case of dyscalculia).