[typeiierror]

The sample selection / non-response bias highlighted in this write-up is a _Big Idea_ problem I've been thinking about recently:

  Limitations...*Trust in surveys and political leanings:*
  About 95 percent of people contacted for the panel chose not to participate because of lack of trust in having a third-party application installed on their computer or other concerns for privacy.

Think about that - a reputable, privacy-first organization asked people to opt-in to fully consented, voluntary, compensated research and ~95% declined! I can't even imagine what hidden skews are present in the 5% that agreed. This issue is systemic in consumer research and impacts both public (e.g. election polling, U.S. census) and private (pharmaceutical trials, media/advertising research, voluntary AI/ML training daat) polling.

Governments and businesses make biased, potentially discriminatory decisions if a non-random segment of the population chooses to never be counted. The ad industry attempts to circumvent this through non-voluntary passive tracking, which trades off non-response bias with bulldozing user privacy. The headwinds are only growing too, as consumer awareness of privacy lapses and the politicization of polling continues to reduce who participates in opt-in research.

Finding a solution to this that doesn't resort to privacy-eroding tactics is a moonshot level problem in terms of the size-of-the-prize if solved.

[nitwit005]

An organization people had never heard of asked them to install a browser plugin, and they declined like they should.

Even if it's an organization you recognize, verifying it's not someone using their name for some sort of scam isn't always straightforward.

[danShumway]

Right. An alternate take on this passage is, "95% of contacts exercised basic security measures rather than blindly install unvetted code on their machine."

It doesn't take that much of a shift in perspective to turn this into a very optimistic statistic indicating that normal users aren't always quite as security-unconscious as we normally think -- that for whatever reason (security, apathy, paranoia, whatever) sometimes they do actually make the right choices when interacting with sensitive information like their browsing habits.

You could not pay me to install an unvetted Electron application where I can't even see the source code, that is designed to MITM my browsing activity. Even if I trust the author's intention, who wrote the app? Who tested it? How do I know that the automatic redactors are going to actually work? It's not like it's hard to have security leaks in Electron.

[kleiba]

But that's exactly the point of the OP: 95% "declined like they should" -- but what does that say about the 5% that didn't? What general conclusions can you draw from data elicited through people that are clearly unlike the mainstream?

[pjc50]

The key thing here is trust. Trust is a resource like a rainforest: you can exploit it sustainably, or you can get a far greater profit by destroying it. Trusting people are a resource that can be exploited for fraud, which the internet is great at producing. It's not really surprising that random organizations find low trust.

It is however very unfortunate, as historically being a "high trust society" has been a great advantage of the west. And it's going to take a lot of repairing.

[Nextgrid]

Is it a problem that should be solved? Shouldn't people have the right to live in peace and not be forced to participate in some survey?

I also don't really see the problem with 95% of people declining; they made the smart choice. If I was a regular Facebook user I would also decline because the account would contain tons of very sensitive data such as DMs and running an untrusted, unknown application on my main computer is also a major dealbreaker.

[typeiierror]

Try applying the problem to other issues to see the impact:

* An advertiser wants to place ads on sites / tv networks that have an audience that is more likely to buy their product upon seeing their ads. If they don't want to violate privacy, they run a survey. What if the response rate among a historical disenfranchised group (e.g. African Americans) is terrible? The modern "data driven" marketer would see little reason to advertise on Black media properties. This isn't a fictious example - it's a current problem in the media planning / agency industry.

* A local government has to decide between investing in more ESL resources in public education vs. other competing budget needs. They look at census / community survey data (which some Hispanic and immigrant populations are fearful of responding to d/t politicization) and decide to prioritize other asks due to undercounted demand. The data could also be skewed in other ways that warp their decision, like allocating budget to school zones that only represent specific immigrant communities that haven't historically been disenfranchised.

The big picture issue here is governments/businesses making decisions with bias information leading to incorrect conclusions, and the only know recourse currently is to scrap privacy.

[TeMPOraL]

Look at it from the point of view of regular folks:

* An advertiser - a malicious being intent on tricking me out of my money - wants to make a survey to determine how to make it easier to trick people into parting with their money. Why would I help someone make my life, and life of other people like me, worse?

The answer to that is to beat advertising down until it isn't so blatantly customer-hostile. Then people may be more willing to help.

* I'm in a politically precarious situation and the government is asking questions - ostensibly for purposes that could benefit me, but if my honest answers were seen by a different government agency, it would cause me a world of hurt. I hide away. Or lie.

The answer to that is ideally to fix the politically precarious situation of a subset of your population - but at the very least, to foster the trust in information separation between government agencies, so that I can e.g. afford to be honest with the census bureau without worrying about the IRS or the police. That level of trust is not the default.

[indymike]

You've really summed up the state of the world right now: we're in a crisis of trust. We don't trust each other, we don't trust institutions and the result is anxiety, fear and anger.

[ryukafalz]

I broadly agree, but I would frame it a bit differently: we have a severe lack of trustworthiness in our modern world; or, at least, the trustworthy voices are lost in the noise.

This is a big big part of why I primarily use FOSS as much as possible. Generally speaking, FOSS developers and distributors seem to act with the user's interests in mind more often than proprietary software vendors. (Certainly the distributions do, probably out of necessity - there's no shortage of competitive distro options, so a distro being shady is practically a death sentence. Individual developers still deserve more scrutiny.)

The advertisers certainly do not have my best interests in mind.

[danShumway]

Agreed. The issue isn't that people don't blindly trust advertisers and VC-backed companies enough. The issue is that those entities are not trustworthy.

People who are choosing not to share data with those companies in their current form are making a smart choice.

[TeMPOraL]

I think you may be underestimating the problem with your framing. The real trouble is outside of software.

If you think about it, how come otherwise reasonable people become anti-vaxxers, or flat-earthers, or believers of any kind of (perhaps less obvious) nonsense? The arguments I've seen tend to boil down to lack of trust. They don't trust healthcare institutions ("it's all bought out by big pharma!"), scientists ("all bought out by big $something!"), government agencies ("they're incompetent"/"literally nazis"), etc.

To some degree, these institutions all violated our trust in one way or another, and media (both mainstream and social) is doing stellar job at amplifying the damage. To me, the problem with the people mistrusting institutions to the extreme isn't the facts - they often have good, if cherry-picked ones. It's the relative weight given to those facts (like, just because there was a screwup with the swine flu vaccine doesn't mean flu vaccines in general are evil dangerous pharma moneymakers). Fixing that requires teaching people some rational thinking, and I'm not sure how to do that; it's much more difficult than just throwing citations at them.

[danShumway]

> and the only known recourse currently is to scrap privacy.

I agree that low response rates are a problem, but people should still have the choice whether or not to give this information. To me, when I see that voluntary participation in these studies is so low, that's not a problem with privacy, that's a problem with the institutions doing the collection.

A good example of that is political surveys, which are really hard because people don't answer their phones. But why don't people answer their phones? Because they're swamped with scams, political ads, and other spam. Half of the time that someone says they're conducting a political survey on a phone call, what they're really doing is campaigning for a candidate.

The problem isn't that people are allowed to decline phone calls, the problem is that most of the phone calls people get are unwanted crap -- so it really doesn't make sense for them to answer the phone, they're making the correct choice by letting unrecognized numbers go to voicemail.

As a further analogy, if 50% of mail in the US postal service was infested with live spiders, you might see delivery rates for paper bills and official notices plummet. That would be a problem. But the solution wouldn't be to force people to open their mail anyway, it would be to stop putting spiders in people's Amazon boxes. And as it is with spiders, so too it is with advertisers.

You want to improve voluntary participation rates? Focus on removing bad actors and making people feel safe about their data. Governments, telemarketers, political groups, advertisers, and just companies in general all have serious issues with self-policing how they use and collect data. That's not anyone else's fault or problem to solve.

[Nextgrid]

> A good example of that is political surveys, which are really hard because people don't answer their phones. But why don't people answer their phones? Because they're swamped with scams, political ads, and other spam.

But why should people answer political surveys? It's a waste of time similar to the other nuisance calls you mentioned.

Even if we assume all the other nuisance calls are eliminated, there's still no reason anyone should answer a political survey. It's a waste of their time and there is no way to ensure how this data will be used.

[danShumway]

It would make elections a bit less stressful. There are a lot of vested interests in both political campaigns and in the public at large that want accurate polling before elections. That's extremely difficult to do right now.

I know some people debate whether having that information is healthy, which I won't comment on, but I do understand why someone might want it.

Now, at an individual level, what do I personally get out of answering any specific survey -- that's a much tougher question for me to answer.

[ForHackernews]

Just do what Nielson does: Pay people for their data.

I don't think it's that difficult; if pay-for-survey skews results toward overvaluing the opinions of the poor and/desperate-for-money, well, then it would be the first time in history.

[AlexandrB]

In my opinion, this is a symptom of weak/ineffective regulation in the personal information space. The consequences for data breaches to the guilty parties have been minimal at best. Meanwhile responsibility for fraud has been pushed onto individuals via concepts like "identity theft". Even if the company in question was indeed reputable and well-known, most people don't have the technical expertise to evaluate any claims about security or privacy. Who would take that risk knowing that at the end of the day most of the consequences will fall on them personally?

[nxpnsv]

Meanwhile fb has all of this info from the 100%...

[OneGuy123]

> a reputable, privacy-first organization...

Are you a shill for them? "Reputable" as mainstream media? "Reputable" as in the fake news is reputable?

Whenever someone claims "I am reputable" you should run away as fast as you can.

This "I am reputable" is purely subjective based on your own biases and incentives.

No one can be reputable in the news space and people must understand this as soon as possible: everyone lies, even natural science which is supposed to be the gold standard gets so many studies wrong.

The only way news organizations stay alive is either by: a) clickbait articles which eventualy devolves into lying or exaggeration at best. b) News orgs that are financed by private people/corporations who have their own agendas.

There is no such thing as a "reputable" news source.

Perhaps one in ten thousand jounralist is still legit, so that he is an actual investigative journalists. But 99.999 of "journalists" are actually just script readers and clickbait writers.

Do you not see where "I am reputable" leads to? Soon the "I am reputable" organization will get political power and then they will make laws based around "my reputable reporting" and this will lead to censortship.

"I am reputable" always leads to censorship down the line since it implies that "my opponent is not reputable and is lying and MUST BE SHUT DOWN IN THE NAME OF DEMOCRACY".

[pjc50]

Per my answer above: the situation where nobody can be trusted is horribly unstable, because evidence-based or impartial policymaking or even justice becomes impossible. This tends to result in replacing trust relationships with force relationships, and the society devolves into warlordism or dictatorship in order to restore order and control. You can't expect people to trust an election where all candidates are disreputable, so they vote in a dictator.

This is why the unreliability of news organizations is such a serious problem.

[skinnymuch]

Look at your response. Why would OP be a shill when you’ve responded this way? It’s not hard to see people will have strong opinions on things even if they believe their opinion isn’t wild, like yourself.