irreproducible_researchA large structure can’t stand without a solid base. It’s the same with drug discovery and irreproducible research. Image credit: Wikimedia Commons

Drug discovery, ever a challenge, can be likened to a tall structure. The upper stories rely on the stability of the floors below it, and a solid foundation is a must. No lofty heights will be reached when the base is not secure.

And so, it is not a surprise that a drug discovery project will collapse if its own base is, well, baseless. Unfortunately, this is not just a hypothetical scenario: There is an increasing realization that many published early stage investigations cannot be repeated and causing the rise of irreproducible research. In one study that underscores the extent of the problem, scientists at Amgen attempted to reproduce the findings of fifty-three “landmark” articles – and only succeeded with six.[1]

What gives? Although it happens too often that a so-called research “finding” in fact stemmed from fraud, it is more often the case that irreproducible research results are caused by rushed, less-than-careful work by scientists under extreme pressure to publish, particularly due to funding and job security concerns. With a weak job market and tightened federal budgets, the pressure has been especially intense. Wait too long to announce results, and another group might reach the finish line first and “scoop” a project. In such an environment, researchers may pay less heed to internal doubts that call for further experiments. As Richard Feynman said, “The first principle is that you must not fool yourself, and you are the easiest person to fool.”

And even when an experiment is performed meticulously, crucial details might be left out of the manuscript. An experiment might only work in a narrow range of temperatures, for example, and others trying to repeat the experiment could flounder without this information. The writers of the original report may feel that there is not enough room in an article to list all the details, or, more nefariously, they don’t want competitors to learn the secrets to their success, and thus continuing the chain of irreproducible research.

Alarm bells are ringing. Nature – one of those top-tier journals that scientists break their backs to get into – wrote about the problem of irreproducible research in a series of articles.[2] Many concerned writers have weighed in about what to do. Although there are disagreements, one common theme emerges: transparency.

Many in the scientific community feel that journals should insist that authors provide every relevant experimental detail in a manuscript. A slight detail can mean a world of difference. Both positive and negative data should be reported, rather than trying to present a finding in the most exciting way possible. Some in the community have even suggested that at least part of the peer review process should be open to the readership. With questionable studies making it through peer review, it has been wondered if the quality of peer review suffers from its secrecy.

People in a position to award grants, tenure, promotions, and job offers can help by looking beyond an applicant’s number of publications in high-impact journals. What is the real quality of the research? After all, irreproducible research is not quality research. And those performing the research should remember that they are less likely to avoid embarrassing corrections and retractions if their experiments are well-rounded. Dedicated service providers through Assay Depot are available to assist. Irreproducibility in research has many causes and no simple answer, but making repairs is imperative. Drug discovery depends on it.

References

[1] Begley, C.G. & Ellis, L.M. 2012. Drug development: Raise standards for preclinical cancer research. Nature 483:531-533.

[2]  http://www.nature.com/nature/focus/reproducibility/

Posted 10:36 am,

“Innovation” is suddenly the word on everybody’s lips.  Pfizer launched “Centers for Therapeutic Innovation”, Johnson and Johnson followed with their “Innovation Centres” and now Merck followed with their “Innovation hubs”.  In fact, it happened so fast that many of us in the dusty halls of academia are still trying to get a grip on what’s […]

Read More...
Posted 1:06 pm,

Sometimes, in tragic circumstances, a lesson can be learned and applied in unexpected ways. That appears to be the case with a rare disease that produces debilitating symptoms in children – but might also offer clues to developing a new class of antivirals. The particular disease, an extremely rare congenital disorder called CDG-IIb,  affects the […]

Read More...
Posted 9:52 am,

We have recently discussed here how crowdfunding — donations from the general public in support of a cause, idea, or invention — has caught the attention of scientists seeking financial backing in an era of tight budgets. The brave new world of crowdfunding in science raises questions that defy easy answers. What is essential to […]

Read More...
Posted 12:28 pm,

Connecting academic scientists to the global life science ecosystem Like everyone else, academic researchers are finding that funding sources are drying up and that they somehow must do more research work with less money. To do something about it, a group of Universities in Missouri and Kansas have teamed up with Assay Depot to create […]

Read More...
Posted 11:39 am,

As we previously discussed, CRISPR-mania is sweeping the globe. (Okay, it might not be as big as Beatlemania, but more than a few life science researchers have probably squealed in delight.) The CRISPR system, derived from bacteria’s fight against viruses and tailored so that it can snip away genetic material in other organisms, has generated […]

Read More...
Posted 2:26 pm,

It’s a sad story that everyone is familiar with: A patient takes one or more well-known cancer drugs, and they work — for a while. However, after the respite, the cancer comes roaring back, this time impervious to the once powerful effects of the cancer drugs. Many people would view the disease’s return with resignation […]

Read More...
Posted 1:52 pm,

Over the course of this series so far we’ve looked at the role of the microbiome in two of the three major therapeutic areas targeted by drug discoverers.  Now comes the third and final segment and it’s a big one:  Cancer.  For many people the role of the microbiome in inflammation and metabolism is unsurprising.  Your […]

Read More...
Posted 4:08 pm,

The vast majority of compounds that enter clinical trials fail to make it through. The result: few drugs, plenty of bills. The consensus in drug discovery is that things need to change. Part of the problem is the maze that is human biology. Choosing the “right” drug target is difficult when 1) information is incomplete, […]

Read More...
Posted 11:07 am,

Outsourcing has proven to reduce costs while dramatically improving productivity of the drug discovery process. Primary benefits to outsourcing drug discovery include higher quality services, direct cost savings, reduced time to clinic, and enhanced innovation. [1] Case in point: Anika Therapeutics recently won FDA Pre-Market Approval for the new osteoarthritis treatment, Monovisc. One of the […]

Read More...
Posted 10:55 am,