In the couple years I spent doing biology research, about 8 years ago, I was skeptical of most research from China because it already had a reputation already of being untrustworthy. But why stop at China?
I spent a good part of a year trying and failing to replicate an English study, and looking around for materials referenced in another study that simply don't exist, finally being told by the principal on that study that they mischaracterized the materials in the paper. Sometimes when you get the results you are hoping for, you publish, and you don't care if they were wrong, right?
Perhaps it would be better if scientists generally took a more skeptical view of others' publications and tried to replicate them in house. At conferences, instead of asking, "Where do we go from here", with the first study, ask, "What do I need to do to replicate this?"
I had a very similar experience. And you are correct that these problems are not at all limited to "China", although those authors tend to be a little less sophisticated in their (what may generously be called) shortcuts.
For example, if you do a real literature review on a medical topic (not just credulously take their word for it) you are almost sure to find that one of the primary assumptions/claims was actually never studied. One paper refs a second which refs a third that speculated about it in the introduction, but now "everyone knows it".
So true. This is especially important as people study ways of improving metrics of markers of disease instead of actually improving disease endpoints. For example, there is a supplement marketed in the veterinary space that may lower blood urea nitrogen in uremic animals, according to a study. Whoop-de-do! I don't care about urea, it doesn't cause the disease of uremia. It is just a marker. You could convert all the urea to ammonia and now have a patient with two diseases and a BUN of 0.
Considering past occurrences https://en.wikipedia.org/wiki/2008_Chinese_milk_scandal and general behaviour in online market places (my last experience was that the product was advertised to be shipped from Germany (expect fast delivery), but arrived a month later from China) then it should not come as very big surprise (I am not saying that this is normal or acceptable, but that it is not very surprising).
My question is that what is the cause of this general fraudulent behaviour? Does it have some historical roots? Does it come from lack of actual functioning oversight?
Another anecdotal case: try asking for directions in China. Even when person does not know the correct directions, they do not mind to direct you, and when confronted, they refuse to admit that they do not know. Is this mindset result of constant fear of repressions, or it runs deeper into the history of the society (say it is also present in Taiwan?).
I am not accusing anybody in anything, I just would like to understand.
Historically scientists are lonely people who are (mostly) motivated by seeking truth or knowledge and satisfying curiosity instead of money or fame. Most people are practical and are motivated by money or fame more than satisfying curiosity. So most people are pursuing jobs rather than science. Fraudulent behavior is simply a result of seeking shortest path (shortcut) to a goal. When the goal is (short term) money and fame, and when the gain of taking the shortcut of fraud is bigger than the negative impact, then the popularity of the short cut is well expected.
So the problem is we reward so-called scientist too much (in terms of money or status). Scientists were historically not rewarded by money or status. In fact I don't think there is traditionally a career of science. People then were doing science on the side of their regular jobs (such as engineers or teachers or nobles who don't really need a job). Only when the reward of doing science is detached (to certain degree) from the reward of money and fame, can we truly prevent the fraudulent behavior.
On the other hand, the allure of money and fame probably can never be truly detached for any normal human, so fraudulent behavior is part of the human nature. It ebbs and flows. There is no need to make a big fuss about it.
PS: from what I heard, China today has so much (relative to its capacity) science money floating around and that got to distort the goal of science significantly.
I would have a hard time swallowing the argument that we pay scientists too much, or that society respects them too much. A distant observer might think this could be the case since, from their perspective, what they see as "scientists" are celebrated professors, showered with job stability, upper-middle-class incomes and book deals. But those lucky sods are vastly outnumbered by postdocs and tenure-track faculty, who work thankless hours at minimal pay and who could put their brains to work anywhere else for better reward. Those are the "typical scientist," on average, these days, and if we paid them any less they could barely afford to eat.
The problem is the imbalance- a postdoc really really wants to get tenure someday, because the only alternative is for his career to peter out and to end up working ignominiously in industry (as a second-rate software engineer, most likely.) And the number of tenure positions that could possibly be available is an order of magnitude fewer than the number of smart people competing for them. So he'll possibly "cut corners" to get there, despite the devotion to his field that made him choose that career and not something reasonable, like finance or software engineering. Academia is tenure or bust- there's no stable middle option.
There is a correction under way as people start to realize that academia is no longer a feasible career path, in the same way that the NFL isn't. So I'm hopeful that things will stabilize in regards to scientific integrity regardless- but, then we'll have to figure out who is going to do the science when we can't rely on an army of poorly paid geniuses.
I won't deny the reality of modern day science worker. But the fact of they are paid/respected at minimum does not justify their motivation. Just like a typical minimum wage worker is not doing the work out of pleasure, a typical postdoc is not pursuing a career of pure knowledge. Of course there are some component in it, but from what I see, they are pursuing a career -- a tenure as you put it, an end result that is secure in income and high in status of respect and rich in freedom -- even when that is more of an illusion. So the bottom line is I don't believe there are as much people as many scientists today that are motivated by curiosity alone. They are pursuing money and fame, even when that is an illusion (to certain extent).
The current science career is more of an engineering career, where people have clear goals with constant feasibility assessment and motivated and encouraged to seek short cuts.
So I am not saying we are paying scientists too much. I am saying scientists don't need to be paid, only need to be sustained. Not a many people would be happy for a career that is merely sustained, but not many people are truly born to be a scientist -- think about Einstein being happy at a patent office.
EDIT:
So I agree that look at the way today we do science, it is of concern. However, I think that is largely a mislabeling. Today's science career is more of a engineering practice; and look at the way we do engineering, we are doing fine today. Some place some people build a shoddy bridge, it is something of gossip, but not much of concern.
Think about it: science is not supposed to produce a product (medicine in this context) but it is supposed to answer some questions (not given by the society but of one's own). To answer a demand or solve a question (with a belief it can be answered), that is engineering.
Of course, engineering is important. And there is nothing wrong for a few people pursuing science on the side while doing their engineer jobs.
Ok, the vision of scientists as an ascetic class only appealing to the dyed-in-the-wool seekers of knowledge has some appeal. But, there's a problem there, too- sure, Einstein would still have published his breakthroughs, but many of the thousands of researchers whose works were the basis for his wouldn't have. The pace of scientific progress we're accustomed to now is the product of the work of thousands upon thousands of scientists; the contribution of celebrated geniuses like Einstein are dramatic but represent a rounding error compared to the whole rate of progress. We need a way to recruit and retain the rank-and-file ordinary-human researchers as well, and that requires compensation somewhat proportional to the effort they're putting in.
I wouldn't go out and say Einstein is a rounding error :).
When we say science progress is built on top of giant's shoulder, that giant don't have to be and often is not just other/previous scientists. Engineering, culture, or even witch crafting all have contributed to science.
We need a lot of rank-and-file ordinary-human doing their ordinary works out of ordinary motivations, they will provide the basis necessary for the a few true scientists to question the known and explore the unknown.
Chinese nationals were in the top 2 of both measures of dishonesty (which didn't correlate all that well in general, so I'm not sure "honesty" is all that easy to measure), while British nationals were in the bottom 2 of both. One of the correlations they found was that rates of honesty were correlated with rates of Protestant Christianity.
It would be interesting to see results of that game played for a million dollars. You go into a room alone and flip a coin. If you flip heads you win the million dollars. If you flip tails you win nothing.
It's also interesting that in situations like the Wells Fargo scandal, honesty didn't even matter. If you were honest, you were simply fired.
Face. In China (and Asia in general) it is perfectly acceptable to LIE in your face if the alternative could be embarrassing. Dont know something? Lie. Competition is better? Lie. Will miss the deadline? Lie with a wide smile while shaking your hand. Lying itself is _not_ embarrassing.
> general behaviour in online market places (my last experience was that the product was advertised to be shipped from Germany (expect fast delivery), but arrived a month later from China)
Sounds more like a drop shipper reselling stuff from Aliexpress or similar, not necessarily anything to do with a Chinese seller. They add a hefty markup and ship directly to you. It's particularly popular with counterfeits ("replicas").
Radio Free Asia (RFA) is a private, nonprofit international broadcasting agency of the United States government[2] that broadcasts and publishes online news, information, and commentary to listeners in East Asia while "advancing the goals of U.S. foreign policy."
This is good catch, but considering the history of such programs (https://en.wikipedia.org/wiki/Radio_Free_Europe/Radio_Libert...), it is that they publish rather information that has been suspended by local governments than something that has been made up.
Of course you should be cautious with what they publish, but this applies to every published information.
data gathered during clinical trials were incomplete, failed to meet analysis requirements or were untraceable, the paper cited a source in the agency as saying.
Outside of China, clinical data is often not shared, suppressed when it does not support the claims, and p-value hacking is used to justify 'off-label' use. This is not usually spun as fabricated, and in commercial drug development my intuition is that 80% of proposed drugs having at least one of these features would be a plausible hypothesis
In chinese culture, losing one's "face" is one of the worst things that can happen to a person. I'm wondering whether the fact that 80% of data has been fabricated is a result of that. As in, "I have to make my findings significant to not lose my face", or, "Presenting negative findings to my superior will make me lose my face". Maybe someone more familiar with this culture could elaborate.
It's not about the face. It was clearly written there "local pharmaceutical companies trying to produce Western drugs struggle to turn a profit"... What about Theranos in the US? Same..
Isn't it because the turn everything into profit within two years "success or nothing" attitude?
If something ruins the world, it will be two countries that under the surface is way too similar to each other.
That seems to be backward. Doesn't "losing face" come from being caught doing the wrong thing or being dishonest, so that the incentive to fabricate data for something as important as scientific research should be very, very low?
I'm not sure that reputational risk is the same thing; I suspect one way to look at it is that a researcher who returns the null case is disrupting the organization by putting a lot of other peoples' work and assurances at risk. So there would be a significant amount of implicit social pressure to show results and maintain organizational harmony.
No, because losing face is not really about morality or honesty in the first place - it's more important than honesty. The primary thing here is the sharp and pervasive divide between facade and reality. The facade is more important. There are a lot of cultures where fronting is the main thing. It's just particularly evident in China.
Also in Japan, as even a nuclear disaster isn't impetus enough to forego 'saving face':
On July 5, an independent investigative commission established by the Japanese Diet issued its final report on the accident at the Fukushima Daiichi Nuclear Power Station. Hailed as the definitive word on the subject thus far, the report points to what it calls the "fundamental causes" of the disaster, all of them cultural. The chairman's message in the report assails "the ingrained conventions of Japanese culture: our reflexive obedience; our reluctance to question authority; our devotion to 'sticking with the program'; our groupism; and our insularity." The nuclear disaster, in short, was "a disaster 'Made in Japan.'" News media around the world characterized the report's damning indictment of Japanese culture as unusually candid for the nation known to do anything to save face.[http://thebulletin.org/japan%E2%80%99s-culture-culprit-nucle...]
Very interesting analysis. If this is accurate, they should get rid of that attitude, pronto, as they have the highest potential of being on the scientific forefront(or at least that of medical science).
Science is having an astounding crisis -- I don't think there's been anything like it in history before.
If this is true about Chinese clinical data? More generally, if there is any simple tag that covers fraud at this scale? Other people working under that tag will experience deleterious effects for years to come.
The only way I see out of this is a rigorous, self-imposed body of standards. Even then I'm not sure that this will be sufficient.
Think of all those people, all of that money, all of the hundreds of thousands of hours of research. For nothing. Actually, it's worse than nothing because it's been flooding the system with fake data.
It's not only about fake data, it's also, maybe even most importantly, the low quality data. Publications with low data quality decrease the signal-to-noise ratio, leading to a slower scientific progress. Purely from a scientific standpoint: If you identify fake data, you just take them out of your model training set. But identifying low quality data is much harder.
A couple of years ago, as this story started to break, I would mention it to scientist friends. "Don't worry," they'd say, "We always have the meta analysis to fall back on."
I don't hear that anymore.
ADD: A critical issue here, which you touch on tangentially, is the mix of motivation and milieu. If I'm making stuff up -- bad data -- other people can identify it and delete it. But if my goal is to appear to be doing good science instead of actually doing it, then it can become extremely problematic both to identify and remedy what I've done.
A lot of "normal" science is done around the margins, with not-so-incredible hypotheses and pedestrian-looking data. Bad science with really poor data fits seamlessly into that model without having any distinguishing characteristics.
It mentions 1622 current clinical trials. Of those, how many would be expected to be approved in a rational system?
I wonder if the huge %age is due to a lot of low-quality applications mixed in with the serious ones. If the barrier to apply is sufficiently low, you would expect a Sturgeon's law dynamic.
> I wonder if the huge %age is due to a lot of low-quality applications mixed in with the serious ones.
I doubt it. Most clinical trial data in the U.S. is similarly fraudulent, but there isn't really any concept of high-quality vs low-quality applications like you might see with YC or whatever.
Instead, pharma companies tend to rely on the same set of 20 or 30 tricks to take chemical compounds that might be no better than placebos and make them look like wonder drugs. I've been keeping a tally of these tricks on my website:
I spent a good part of a year trying and failing to replicate an English study, and looking around for materials referenced in another study that simply don't exist, finally being told by the principal on that study that they mischaracterized the materials in the paper. Sometimes when you get the results you are hoping for, you publish, and you don't care if they were wrong, right?
Perhaps it would be better if scientists generally took a more skeptical view of others' publications and tried to replicate them in house. At conferences, instead of asking, "Where do we go from here", with the first study, ask, "What do I need to do to replicate this?"