Thursday, February 4, 2016

Amgen publishes failures to reproduce research: So what are they really saying?

From Nature:
Right now, the main way that the scientific community spreads the word about irreproducible research is through innuendo, which is inefficient and unfair to the original researchers, says Ricardo Dolmetsch, global head of neuroscience at Novartis’s Institutes for Biomedical Research in Cambridge, Massachusetts. “Anything we can do to improve the ratio of signal to noise in the literature is very welcome,” he says.
Totally agree. Finding the right people to tell you that a particular R&D approach is irreproducible because of factors beyond your control is a difficult, time consuming, and potentially expensive process. So why aren't there more negative results published?
Partly, because negative results don't sell too well to a community of people interested in positive results. But I think a strong disincentive to publish these 'contra-papers' is a very personal one:
Academic researchers are unlikely to risk alienating their peers by publishing disconfirming results, predicts Elizabeth Iorns, head of Science Exchange in Palo Alto, California. 
So why is Amgen now taking this role?
The idea emerged from discussions at a meeting focused on improving scientific integrity, hosted by the US National Academy of Sciences in 2015. Sasha Kamb, who leads research discovery at Amgen, said that his company's scientists have in many instances tried and failed to reproduce academic studies, but that it takes too much time and effort to publish these accounts through conventional peer-review procedures.
I bet somewhere, somehow, someone has run the cost-benefit analysis between spending 'many instances' of resources chasing scientific geese and the cost of drawing metaphorical borders around impractical research findings (I won't say 'bad', because they may actually be useful in a non-industrial setting).

The balance of the analysis was probably strong enough to make Amgen send the following message, in a very nice way: "In the future, if you don't think we can reproduce your research, don't waste our time it".