Study can’t confirm lab results for many cancer experiments
Eight years ago, a team of researchers launched a project to carefully repeat early but influential laboratory experiments in cancer research.
They recreated 50 experiments, the type of preliminary research with mice and test tubes that is paving the way for new cancer drugs. The results reported Tuesday: About half of the scientific claims did not hold.
“The truth is, we are wrong. Most of what we claim to be new or significant is not,” said Dr. Vinay Prasad, oncologist and researcher at the University of California at San Francisco, who was not involved in the project. .
It is a pillar of science that the strongest findings come from experiments that can be repeated with similar results.
In reality, researchers have little incentive to share methods and data so that others can verify the work, said Marcia McNutt, president of the National Academy of Sciences. Researchers lose prestige if their results do not stand up to scrutiny, she said.
And there are built-in rewards for posting findings.
But for cancer patients, it may spark false hopes to read the headlines of a mouse study that appears to promise a cure “just around the corner,” Prasad said. “The progress of cancer is always slower than we hope.”
The new study reflects on gaps early in the scientific process, not on established treatments. By the time cancer drugs hit the market, they have been rigorously tested on a large number of people to make sure that they are safe and effective.
For the project, the researchers attempted to repeat experiments from articles on cancer biology published from 2010 to 2012 in major journals such as Cell, Science and Nature.
Overall, 54% of the original results did not meet the statistical criteria defined in advance by the Reproducibility Project, according to the team’s study published online Tuesday by eLife. The nonprofit eLife receives funding from the Howard Hughes Medical Institute, which also supports the Associated Press’s Department of Health and Science.
Among the studies that failed, one found that a certain gut bacteria was linked to colon cancer in humans. Another involved a type of drug that reduced mammary tumors in mice. A third was a study in mice of a potential prostate cancer drug.
A co-author of the prostate cancer study said research at the Sanford Burnham Prebys Research Institute has stood up to further scrutiny.
“There is a lot of replication in the (scientific) literature of our findings,” said Erkki Ruoslahti, who has started a company that is currently conducting human trials on the same compound for metastatic pancreatic cancer.
This is the second major analysis of the Reproducibility Project. In 2015, they encountered similar problems when they tried to repeat experiments in psychology.
Study co-author Brian Nosek of the Center for Open Science said it may be pointless to move forward without first doing the work to repeat the results.
âWe start a clinical trial, or we create a start-up, or we trumpet to the world ‘We have a solution,’ before we’ve done the follow-up work to verify it,â Nosek said.
The researchers tried to minimize the differences in the way cancer experiments were conducted. Often they couldn’t get help from the scientists who had done the initial work when they had questions about which strain of mouse to use or where to find specially designed tumor cells.
“I was not surprised, but it is concerning that around a third of scientists have not been helpful and in some cases have been more than that,” said Michael Lauer, deputy director of extramural research at the National Institutes of Health.
The NIH will attempt to improve data sharing among scientists by requiring it from grant-funded institutions in 2023, Lauer said.
âScience, when done right, can produce incredible things,â Lauer said.
For now, skepticism is the right approach, said Dr Glenn Begley, a biotechnology consultant and former head of cancer research at drugmaker Amgen. Ten years ago, he and other inside Amgen scientists reported even lower confirmation rates when they attempted to repeat published cancer experiments.
Cancer research is difficult, said Begley, and “it is very easy for researchers to be drawn to results that seem exciting and provocative, findings that seem to further support their favorite idea of ââhow cancer should be. work, but which are just plain wrong “.
The Associated Press’s Department of Health and Science receives support from the Department of Science Education at Howard Hughes Medical Institute. The AP is solely responsible for all content.
Copyright 2021 The Associated Press. All rights reserved. This material may not be published, broadcast, rewritten or redistributed without permission.