[hnnewguy]

How did I know that there would be a comment like this at the top, someone criticizing the research for not being science?

What is wrong with making up and using a narcissism metric and seeing what it correlates with? It's how social science is done, and how you discover results that are worth exploring. Get off your high horse.

[gd1]

Erm, because the first thing a real scientist would do would be to test the narcissism metric and check that it does, you know, actually correlate with narcissism? You can't just invent metrics and hope that they work.

> It's how social science is done, and how you discover results that are worth exploring.

Do you see the problem here? If you invent metrics, assume they work, then only pursue the ones that produce results "worth exploring"... can you see the problem?

[ubercow13]

How would you do that check if the metric the author chose was the best he could come up with to measure the quality of narcissism? If you want to check your metric correlates with narcissism, what metric B do you check your metric A for correlation with?

[gd1]

Well, obviously there are more painstaking and accurate ways of determining whether a person is a narcissist. No doubt this is hard work. Sit down, talk with them, ask their employees, etc. Do the really hard work for a subset, check that the subset you've tested correlates well to your made-up metric, and only then can you use the made-up metric. You can't just use it without any evidence it works.

[vacri]

What you're describing is just as 'made up' as the metric you're opposing - your 'hard yards' are still based on people's opinions, politicking, and ulterior motives.

Keep in mind that even the GP's original quote included at least two references to the difficulty in measuring the trait ('tricky' and 'had to use indirect measures'). It's not like the researcher boldly strode forth and said "This is the way, have faith!". The results are also suitably couched. It seems like this "bad science!" thread is making a mountain out of a molehill.

[gd1]

>What you're describing is just as 'made up' as the metric you're opposing

So are you telling me that the very best way we have of establishing narcissism is the "the frequency with which the CEO’s name appeared in company press releases"??

If not, then a scientist is required to either:

a) Use the best method.

or...

b) Develop a proxy and calibrate it against the best method.

You can't just postulate a proxy for narcissism out of thin fucking air and then use it.

It scares me that I have to explain this.

[vacri]

Nice false dichotomy there. Where did I say it was "the very best way"? And I can see just as many ways for your suggestion to be corrupted as the method in question. If it 'scares you that you have to explain this', then you should be offering some actual method that is good as you demand, because what you offered is, frankly, subject to terrible bias on the part of the respondants, and still has the bias of the researcher in deciding what means what.

Your method much less robust than the method in the article, and it might not even get past an ethics committee, for that matter - formally questioning people about whether their boss was up themselves? Think about how could that possibly backfire and significantly harm the subjects. Where would you question them - at the workplace as a job lot? Or would you track them individually outside of work, where these kinds of questions about the boss could now be seen as an organised attempt at harrassment, given the effort required?

This was an exploratory study trying something out, with explicit mention that the measures were indirect. It's not like there's an international treaty being based on it. Calm the fuck down.

[snowwrestler]

It is the easiest thing in the world to read about a scientific study and then hand-wave about how it should have been done better. Actually doing a study is quite a bit harder, for all the typical reasons that things are harder to get done in real life than on paper.

To pick at one example, do you think it is easy to get time to just sit down and talk with a CEO or their employees? Particularly when the subject of your research is how narcissistic the CEO might be?

[hvs]

If that's how social science is done, then the "science" part should be dropped. My guess is that social scientists are (hopefully) more rigorous than that.