research

What science is worth shouting about?

Martin Fenner
January 10, 2009 4 min read

A few weeks ago I wrote about the different ways results of a clinical trial can be reported (What are the right numbers for JUPITER). Inspired by blog posts by Eva Amsen (Failure) and Sally Church (Over hyped cancer drugs or sensational journalism?), I thought more about what makes scientific findings worth talking about outside of your immediate research community. I will look at cancer research, and it is obvious that some research findings are more exciting than others. But it less obvious that it is as important who is communicating the research and what is the intended audience. This is especially true for translational research (Translational Research 2008), where research findings can change the way we treat patients and drug companies (and others) can potentially earn a lot of money.

The academic researcher

Academic researchers involved in basic or clinical cancer research are interested in publications and grant money. As Eva said: the unit of success is the publication record. Basic cancer research can be very exciting and touches many related fields from signal transduction to epigenetics. But what confuses me is when basic research findings are related to the treatment of human cancer, which is a completely different story. Cell lines are not living organisms, mice are not men and very few basic research findings make it into a clinical trial.
Unfortunately less than one in five studies in cancer (they all have to be registered with clinicaltrials.gov since 2005) have been published in peer-reviewed journals (Ramsey 2008). Negative results will usually not be published, and this so-called publication bias creates many problems. Just imagine 5 clinical trials with a new drug where one trial will show an advantage for the new treatment. This trial will be published in a nice journal, probably one negative trial will be published in a small journal and the remaining 3 trials will remain unpublished. Just looking at the published literature will of course give the wrong impression, but these are the only data that are available to most people.

The drug company

A drug company is interested in marketing approval of a drug. This approval is obtained from the Food and Drug Administration (FDA) in the United States and from the European Medicines Agency EMEA for most of Europe. Approval of a cancer drug usually requires one or more large randomized trials (so-called phase III trials) that shows that the drug improves survival. And up to 50% of these phase III trials fail to show the desired effect, even when earlier (so-called phase II) trials were positive (Chan 2008). Satraplatin is one recent example of a drug that showed promising results in the treatment of prostate cancer but failed to prolong survival – and therefore was not approved by the FDA.

The insurance company (or whoever pays for medical care)

The drug erlotinib was approved for the treatment of pancreatic cancer, but the median benefit in overall survival was less than 2 weeks (Moore 2007). Insurance companies are interested in treatments that are not only effective, but also cost-effective. For cancer treatments one can calculate the cost per quality-adjusted life year (QALY). Adding erlotinib to gemcitabine in the treatment of pancreatic cancer would cost about $410,000 per year of life saved (Miksad 2007). Costs over $100.000 per year of life saved are considered high and the high cost is the reason that the British National Institute for Health and Clinical Excellence (NICE) decided to not recommend 4 new drugs for the treatment of renal cancer in August 2008. NICE doesn't question the effectiveness of these drugs, but wants these drugs to become cheaper.

The media

Cancer research sells. Cancer is very common and many people can tell sad stories of friends or relatives that died because of cancer. And we want to hear encouraging stories. But the media often confuse promising findings in basic research or early clinical trials with a new cure for cancer. One of the more famous examples is the statement Judah is going to cure cancer in two years by James Watson in a 1998 New York Times story on angiogenesis research (A Cautious Awe Greets Drugs That Eradicate Tumors in Mice). Another example is this recent article (cited by Sally Church in her blog post mentioned above) in the Times (Cancer drug could save the lives of 10,000 a year). Ben Goldacre has written more than one critical blog post about how traditional media report cancer research (e.g. Still no cure for cancer hysteria).

The patient

A patient with cancer needs a drug treatment that either cures his cancer or prolongs survival. The new treatment should either be better or have fewer side effects than the current standard of care. But many cancer patients can't wait until a new cancer drug is approved and available for prescription. So whenever possible, patients should participate in a clinical trial. Clinical trial participation by adult cancer patients is unfortunately low (less than 5% compared to more than 50% for children with cancer (Sateren 2002)), but this is the best way to receive a promising new drug as early as possible (and to stop receiving it when the drug has been shown to be ineffective or toxic).

Conclusions

We should be careful how we report research findings outside of specialist journals or research meetings, especially if these findings could have important consequences outside our immediate research community. And as a critical reader we should assume that most published research findings are false (Ioannidis 2005).

References

Translational Research: Getting the message across. (2008). Nature, 453(7197), 839–839. https://doi.org/10.1038/453839a

Ramsey, S., & Scoggins, J. (2008). Commentary: Practicing on the Tip of an Information Iceberg? Evidence of Underpublication of Registered Clinical Trials in Oncology. The Oncologist, 13(9), 925–929. https://doi.org/10.1634/theoncologist.2008-0133

Chan, J. K., Ueda, S. M., Sugiyama, V. E., Stave, C. D., Shin, J. Y., Monk, B. J., … Kapp, D. S. (2008). Analysis of Phase II Studies on Targeted Agents and Subsequent Phase III Trials: What Are the Predictors for Success? Journal of Clinical Oncology, 26(9), 1511–1518. https://doi.org/10.1200/jco.2007.14.8874

Moore, M. J., Goldstein, D., Hamm, J., Figer, A., Hecht, J. R., Gallinger, S., … Parulekar, W. (2007). Erlotinib Plus Gemcitabine Compared With Gemcitabine Alone in Patients With Advanced Pancreatic Cancer: A Phase III Trial of the National Cancer Institute of Canada Clinical Trials Group. Journal of Clinical Oncology, 25(15), 1960–1966. https://doi.org/10.1200/jco.2006.07.9525

Miksad, R. A., Schnipper, L., & Goldstein, M. (2007). Does a Statistically Significant Survival Benefit of Erlotinib Plus Gemcitabine for Advanced Pancreatic Cancer Translate Into Clinical Significance and Value? Journal of Clinical Oncology, 25(28), 4506–4507. https://doi.org/10.1200/jco.2007.13.0401

Sateren, W. B., Trimble, E. L., Abrams, J., Brawley, O., Breen, N., Ford, L., … Christian, M. C. (2002). How Sociodemographics, Presence of Oncology Specialists, and Hospital Cancer Programs Affect Accrual to Cancer Treatment Trials. Journal of Clinical Oncology, 20(8), 2109–2117. https://doi.org/10.1200/jco.2002.08.056

Ioannidis, J. P. A. (2005). Why Most Published Research Findings Are False. PLoS Medicine, 2(8), e124. https://doi.org/10.1371/journal.pmed.0020124