We're in an interesting point: there is so much we don't know but in order to learn more everything we do must fit in the already immense amount of knowledge that we accumulated so far. In the vast majority of cases this requires that the people who want to nudge the frontier a bit further must first dedicate a good portion of their life's studying what we know, and as the result sounding a bit arrogant when they explain to a layperson that actually they know what they're talking about. Yes, in some cases, they may be erring on the side of too confidence, but in many many cases is the layperson who doesn't fully grasp the ramifications of the innocent looking alternatives.
That’s a pretty long way around to what is essentially an appeal to authority. You’re right about the knowledge and devotion required of frontier pushers. History is full of people who challenged this thinking and completely overhauled human understanding of a topic, though, often in the face of relentless ridicule.
The error (in your telling) is equating knowledge with confidence. Knowledge is knowing you might be wrong about it all. The advice to spend one’s life questioning isn’t a smarmy nothing; it’s the only truly sensible approach when you step back and think about it.
It's not actually an appeal to authority as such - authority implies (usually) institutional accreditation, whereas here we are pointing out the situation is so complex opinions without years of study are more or less pointless.
Then, when we have two e.g. physicists, who both know quite well what they are discussing, and one of them is more famous and potentially through their prestige succeed in ridiculing their less recognized colleague, we are at the "appeal to auhority" position.
One famous example is that of Ernst Mach who was was positivist (i.e. did not respect theory whose constituents you could not directly measure) and ridiculed Boltzmanns kinetic theory of gases because Mach did not believe in atom theory (!). Boltzmann's theory was effectively attacked precisely from position of authority.
So, if a layman and a physicist argue what is possible, it is very likely while both of them may be wrong, the layman likely does not have any understanding what their position implies.
So in my opinion, you can have a pathological appeal to authority sort of situation only when two equally skilled persons have an argument an the institutional prestige of one of them is used as an appeal for them.
> History is full of people who challenged this thinking and completely overhauled human understanding of a topic
At the risk of argumentum ad logicam, this is a textbook example of survivorship bias. History is also full of people who were adamant they were correct in the face of ridicule, and turned out to be wrong anyway.
Appeal to authority is only a fallacy when the authority is not an actual authority on the exact topic being mentioned.
Your broader point is right; obviously if we stop to poke at certain assumptions, the occasional one will collapse.
However, the pathway you’ve just suggested is less practical than you think. The GP is talking about a systematic, coordinated exploration effort of known unknowns.
Metaphorically - he/she is saying that there’s more likely to be gold at the unexplored end of gold mine, not in the excavated dirt.
What engineers work with is, maybe, 1/1000 of our physics knowledge (maybe 2/1000 for electronical engineers who need a solid basis of quantum mechanics).
Our physics knowledge is maybe 1/1000 of what we roughly know should be there but cannot be probed (quantum gravity, nonlinear field theories, dark stuff...).
The Universe is so huge that it is pretty impossible to descrive how much bigger than us it is - probably infinitely.
The point is, between the stuff that we know and the stuff that we roughly know but don't really know - we know a lot more than what we can use.
Saying that something is not so useful technologically, as OP stated, is rather a safe statement. We know a lot about fermions and bosons, light and electrons - and we know sufficient information to be able to state when something is overhyped and not really useful as it seems