Given the quality of the search results I've seen recently, I feel like Google has made the ranking almost random... it's only within the past few years that I've had a lot more of "I'm pretty sure there's more information on this out there, but Google isn't showing it ot me" experiences.
I've had that feeling too, but I wonder if it's because the good old Internet of enthusiasts enthusiastically posting about the things that enthuse them has given way to the faceless content farms to such an extent that even Google isn't able to filter the signal out from the noise. I.e., I wonder if it is not so much their fault for being less good than they used to be as it is their fault for not being so much better than they used to be.
The end users have changed from enthusiasts who want detail to average people who want to get caught up on a subject quickly. It seems to be two fold and I think google have just given up trying to filter signal to noise and just promote conglomerates that it sees as authoritative.
(Apologies for offtopicness: could you please email us at hn@ycombinator.com? I want to send you some repost invites but don't have an address to do that.)
At best, Google is correlating concrete things to quality. There are plenty of SEO contests where you can see they can still be fooled. Including a pretty famous one where a site that's 100% Latin ranks for a competitive local search query.
Things like load speed, security and mobile responsiveness can make or break a site's Google performance. So yes, some stuff works but chasing trophy keywords and such is a fools errand.
Why do we always assume that Google is some entity with perfect intelligence? Isn't it just a bot? People outsmart it all the time? The very fact that SEO exists means that you can fool Google.