At Open Health Care we believe in Evidence Based Medicine.

Which is why we create digital health services that support research.

Unfortunately the current infrastructure supporting evidence based medicine has some shortcomings. One of these is that when a research paper is retracted because, for example, the author made up the results, there is little or no means to propogate this update in the wider medical literature.

Supposing, as a doctor, I look to a systematic review on the management of heart attacks to inform how I treat my patient, I sadly have no assurance that the systematic review does not rely on research papers that have been retracted. This is suboptimal.

Open access, post-publication peer review, and better version control would be three useful features that a modern scientific publishing model optimized to enable good science should support.

In the meantime, inspired by a lunchtime conversation with our friend and director Ben Goldacre (it was all his idea), we pondered the question:

What is the most cited retracted paper?

It turns out PubMed already has a nice API that can return a list of PubMed ids for papers that have been retracted. Next we needed a means of knowing how many times each retracted paper was cited. There are a few ways of doing this but we went with using google scholar after Alf Eaton told us about it on twitter. It’s possible to search google scholar for the articles citing a particular pubmed id, here’s an example.

Finally, we used wget to download all of the google scholar search results. This infamous paper ‘Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children.’ was the winner with 2070 citations!

Here’s the top ten:

Citations	Title		Author	Journal	Pubdate
2070	Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children.		Wakefield AJ	Lancet	28/02/1998
2050	Visfatin: a protein secreted by visceral fat that mimics the effects of insulin.		Fukuhara A	Science (New York, N.Y.)	21/01/2005
1550	Purification and ex vivo expansion of postnatal human marrow mesodermal progenitor cells.		Reyes M	Blood	01/11/2001
1250	Combination treatment of angiotensin-II receptor blocker and angiotensin-converting-enzyme inhibitor in non-diabetic renal disease (COOPERATE): a randomised controlled trial.		Nakao N	Lancet	11/01/2003
1040	Spontaneous human adult stem cell transformation.		Rubio D	Cancer research	15/04/2005
825	Regression of human metastatic renal cell carcinoma after vaccination with tumor cell-dendritic cell hybrids.		Kugler A	Nature medicine	01/03/2000
805	Evidence of a pluripotent human embryonic stem cell line derived from a cloned blastocyst.		Hwang WS	Science (New York, N.Y.)	12/03/2004
755	Multiple atherosclerotic plaque rupture in acute coronary syndrome: a three-vessel intravascular ultrasound study.		Rioufol G	Circulation	13/08/2002
732	Structure of MsbA from E. coli: a homolog of the multidrug resistance ATP binding cassette (ABC) transporters.		Chang G	Science (New York, N.Y.)	07/09/2001
616	Synergistic activation of estrogen receptor with combinations of environmental chemicals.		Arnold SF	Science (New York, N.Y.)	07/06/1996
607	Contribution of human alpha-defensin 1, 2, and 3 to the anti-HIV-1 activity of CD8 antiviral factor.		Zhang L	Science (New York, N.Y.)	01/11/2002

You can reproduce our results or view our workings over on Github.

There are of course many reasons other than the author making up the results for a paper to be retracted. There are also many circumstances in which citing a retracted paper is legitimate. For a more nuanced discussion of retractions there’s a nice Comprehensive Survey of Retracted Articles from the Scholarly Literature in PLOS and the excellent Retraction Watch covers the details of individual retractions as they happen.

Anyway, it was fun to examine retractions. We’d love to explore this further so if we can be of service to you do get in touch.

Perhaps a project for NHS Hack Day?

Follow Carl on twitter