[MichaelGroves]

>>> Good sociology papers are just as scientific as physics or engineering papers,

> Snobbery about the conventional levels of significance (etc) is almost entirely beside the point.

It seems like entirely the point. Also, the difficulties you describe are reasons sociology can't be as scientific as physics, not reasons to pretend sociology is just as scientific.

[s1artibartfast]

I think the problem is conflating being scientific with numerical. This paper is not very numerical, but is scientific. It lays out observations and a hypothesis that might explain those observations.

[MichaelGroves]

> It lays out observations and a hypothesis that might explain those observations.

The part you're missing here is where those hypotheses are tested. A field that consistently makes accurate predictions is more scientific than one which rarely does, regardless of how much effort is expended. A field being more scientific than another isn't a matter of researchers trying harder. The quality of their results, independent of the magnitude of their effort, is what really matters. If researchers dutifully follow the scientific method by the textbook and, due to the subject of their research, can only get slipshod results, their field is less scientific than one in which researchers easily get high quality results.

[mattkrause]

This definition seems very idiosyncratic to me, and wrong in more than a few ways.

Science is usually defined as a process of systematic exploration, rather than a set of results. One goal of that process is to make predictions, but it’s not the only one. Even if it were, I don’t see how you can aggregate the quality of predictions within and across fields.

Physics is a perfect example. It’s true that some phenomena have been measured with exquisite precision and can be used to make extremely accurate predictions (relativity, say). Other phenomena turn out to be immeasurable: chaotic systems are, by definition, virtually impossible to predict even though they are generated by simple rules. I find those rules more interesting than yet-another decimal place on the fine structure constant. Plus, it seems incoherent to argue that particle physics is more scientific than biology which is more scientific than atmospheric physics but…

Finally, even if science were defined by predictive accuracy and there were a sensible way of comparing them across fields…the alpha levels/confidence intervals from individual experiments certainly aren’t the right way to do it! They’re a mishmash of the intrinsic variability of the thing under study, then resources devoted to studying it, and the assumptions baked into your design. In 2011, OPERA reported a six-sigma detection of faster than light neutrinos. This was, of course, not really true: a loose cable that was not included in their model was responsible instead.

[MichaelGroves]

The scientific process is a means to an end. The only reason it has any value at all is because it gets better results than other processes. But this process is not equally effective in all fields of inquiry. Those fields in which the process is more effective are "more scientific". Fields in which this method are less effective are less scientific.

[s1artibartfast]

>The part you're missing here is where those hypotheses are tested.

There is no rule that every science paper needs to do testing and cover the whole scientific method from start to finish. Some can just be building blocks; data, hypothesis, ect.

>A field that consistently makes accurate predictions is more scientific than one which rarely does

I couldn't disagree more. That just means the field is more predictable