Washington Post
Many scientific studies can’t be replicated. That’s a problem.
https://www.washingtonpost.com/news/...only-36-times/
Maverick researchers have long argued that much of what gets published in elite scientific journals is fundamentally squishy — that the results tell a great story but can’t be reproduced when the experiments are run a second time.
[
No, science’s reproducibility problem is not limited to psychology]
Now a volunteer army of fact-checkers has published a new report that affirms that the skepticism was warranted. Over the course of four years, 270 researchers attempted to reproduce the results of 100 experiments that had been published in three prestigious psychology journals.
It was awfully hard. They ultimately concluded that they’d succeeded just 39 times.
The failure rate surprised even the leaders of the project, who had guessed that perhaps half the results wouldn’t be reproduced.
the economist
Problems with scientific research
How science goes wrong
http://www.economist.com/news/leader...nce-goes-wrong
A SIMPLE idea underpins science: “trust, but verify”. Results should always be subject to challenge from experiment. That simple but powerful idea has generated a vast body of knowledge. Since its birth in the 17th century, modern science has changed the world beyond recognition, and overwhelmingly for the better.
But success can breed complacency. Modern scientists are doing too much trusting and not enough verifying—to the detriment of the whole of science, and of humanity.
Too many of the findings that fill the academic ether are the result of shoddy experiments or poor analysis (see article). A rule of thumb among biotechnology venture-capitalists is that half of published research cannot be replicated. Even that may be optimistic. Last year researchers at one biotech firm, Amgen, found they could reproduce just six of 53 “landmark” studies in cancer research. Earlier, a group at Bayer, a drug company, managed to repeat just a quarter of 67 similarly important papers. A leading computer scientist frets that three-quarters of papers in his subfield are bunk. In 2000-10 roughly 80,000 patients took part in clinical trials based on research that was later retracted because of mistakes or improprieties.
What a load of rubbish
Even when flawed research does not put people’s lives at risk—and much of it is too far from the market to do so—it squanders money and the efforts of some of the world’s best minds. The opportunity costs of stymied progress are hard to quantify, but they are likely to be vast. And they could be rising.
...
scientific America
Psychology's Ongoing Credibility Crisis
http://blogs.scientificamerican.com/...bility-crisis/
It’s a tough time to be a young psychologist. This thought keeps occurring to me as we search for a new psychology professor at my school, Stevens Institute of Technology. When I meet candidates, I have to ask about their field’s replication—and credibility—crisis.
I feel as though I’m pressing them on some sordid personal matter, like whether alcoholism runs in their families, but the topic is unavoidable. Last summer, a group called the “Open Science Collaboration” reported in Science that it had replicated fewer than half of 100 studies published in major psychology journals.
The New York Times declared in a front-page story that the report “confirmed the worst fears of scientists who have long worried that [psychology] needed a strong correction. The vetted studies were considered part of the core knowledge by which scientists understand the dynamics of personality, relationships, learning and memory. Therapists and educators rely on such findings to help guide decisions, and the fact that so many of the studies were called into question could sow doubt in the scientific underpinnings of their work.”
The crisis keeps generating headlines. On Friday, a group of four prominent psychologists led by Daniel Gilbert of Harvard claimed in Science that last year’s Open Collaboration study was statistically flawed and did not prove its claim that “the reproducibility of psychological science is low.” “Indeed,” Gilbert and his co-authors state, “the data are consistent with the opposite conclusion, namely, that the reproducibility of psychological science is quite high.”
Nature magazine
http://www.nature.com/news/over-half...y-test-1.18248
Phys.org
http://phys.org/news/2013-09-science-crisis.html