[doubletgl]

I didn't quite get why the article described these surveys as problematic. I skimmed the linked article in that section but this didn't provide any answers either. Assuming the participants fully understand the question, and the survey is designed well, what cultural aspect is stopping people from answering?

I grant that with a questions like "Do you often do X?", examples are necessary to specify what "often" means.

From the article:

> Some people may refuse to answer. Others prefer to answer simply yes or no. Sometimes they respond with no difficulty.

That just sounds like some people boycott the Likert questions, but we don't know why.

[rando14775]

I don't know what reasons other people have for refusing to answer. As a bit of pedant I just wanted to explain why I personally don't like them, it has a lot to do with being vague, abstract, unclear. Confusing really. You sit there and are not sure you can even answer the question honestly.

In a normal situation I would probably refuse and ask them to clarify, or challenge their assumptions or something. In this situation I just try to get over my aversion to answer and try to interpret it as best as I can. Sometimes I got no clue what it could mean and choose neutral or whatever. It's mentally exhausting.

This doesn't invalidate the methodology. Just saying I don't like it. If people refuse to answer or answer in a way that doesn't fit the schema and they throw it out, that's obviously a problem though.

Edit: To maybe answer your question, I skimmed through the actual article and they mention things like old people not culturally accepting a young person administering questions, children thinking they should not speak in the presence of elders. But also other weird things as mentioned that seem to me as stemming from confusion. I mean, you get better at answering with practice, so the whole concept might be totally baffling to some people which haven't gone through a Western school, where you're also quite often expected to answer unclear and confusing questions on tests.

More edit: The PhD students and professors designing these questions and the college students used as test subjects are basically the most overschooled people on the planet. They are the ones that, in school, excelled at answering abstract, underspecified, questions with multiple choice answers. Many have seen these before. They might even know how they are made and how the results get processed. Of course they're going to have less trouble answering.

[JoeAltmaier]

Agreed. They are like 'which color represents your opinion on this complex subject?' I don't know what to say - I agree with some of it, disagree with another part, and have no opinion about how it applies to others anyway.

Then there are the 'Have you quit beating your wife?' questions, posed to make it a dichotomy but I'm somewhere else entirely.

[doubletgl]

Surveyors and surveyees being in the same pool of people often is probably bad, I agree.

My conclusion is that the usability and accessibility of empiric testing could use some work. Even in regards to WEIRD people :D

[dhimes]

The problem with these Likerts is that they are often an uncalibrated scale, so the subject is interpreting the question as well as the possible responses. They are also used in medical studies of pain, for example. You've seen them- the little faces in the doctor's office indicating differing amounts of discomfort and you pic the one that describes you). My buddy and I had a good laugh thinking up how to calibrate it :)

I'm not even sure how you test something like that for reliability.