[jonahx]

It's true as I noted there is no final god-like arbiter. But that is not really an interesting observation imo. Taking that perspective to its logical conclusion we end up in a world where values are utterly flat and relativist, and the only thing we can say is that we can't say anything about anything.

It's also true the selection-bias you described exists, in some cases to the point of collective delusion. But note how I can say that and you can immediately think of cases that fit and cases that don't...

On balance there is something real and (despite my first sentence) I want to say "objective" in most cases of expertise. In practice everyone lives as if that were true, even if they are arguing otherwise.

Regardless, even if you want to make the most contrarian, relativist case possible, the phenomenon of expertise (simply viewed as a social pattern) does exist and governs nearly every domain where people talk about "quality".