Back in 2012, Doc James made a query over at WikiProject Medicine about what sort of work could be done by a bot to find retracted papers. At the time, there was no centralized way of finding retracted papers, so Rich Farmbrough queried the PubMed database looking for retraction-related keywords (like 'retraction of publication' in the metadata). Of the roughly 4000 retractions, he found 138 that matched papers cited on Wikipedia. The template {{retracted}} was created to flag those papers, and was manually and semi-automatically added to articles.
Then in 2018, JenOttawa noticed the then newly-launched RetractionDatabase.org, a database of retractions maintained by Retraction Watch. This led Samwalton9 to code the first iteration of RetractionBot. The bot was then doing automatically what people did manually, saving everyone a lot of hassle. However, the bot only ran for a few months, and hit a snag: several Cochrane Reviews were flagged as retracted for technical reasons, while they were never retracted in actuality. The bot was put on hiatus, and Samwalton9 never got to fixing the issue.
Five years later, motivated by the slew of retractions hitting major publishers from Elsevier, Hindawi, SAGE, and many others, as well as the opening up of RetractionDatabase.org (now with nearly 40,000 retractions), I thought it would do us some good to kick the hornet's nest and see if I could interest someone in revisiting this project.
Turns out I could. Not even a week after probbing the volunteers at WP:BOTREQ, mdann52 graciously took over maintenance of RetractionBot, and the bot is now back alive, with many improvements. In particular, it now covers expressions of concerns, not only retractions, which are early warning signs that a paper might be dubious and could be retracted/in need of a major revision. This led to the creation of {{expression of concern}}, which works very similarly to {{retracted}} (see below).
The bot first downloads a .csv file containing all the information in the RetractionDatabase (a 50MB download available here). Then it crosschecks retracted DOIs and PMIDs in the database against those found on Wikipedia.
If a match is found, the bot will, for example, change
...Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.[1]
1. ^Restrepo-Arango, Marcos; Gutiérrez-Builes, Lina Andrea; Ríos-Osorio, Leonardo Alberto (April 2018). "Seguridad alimentaria en poblaciones indígenas y campesinas: una revisión sistemática". Ciência & Saúde Coletiva. 23 (4): 1169–1181. doi:10.1590/1413-81232018234.13882016. PMID29694594.
to
...Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.[1]
It is now up to humans like you to review if this is problematic for the article. If the citation is no longer reliable, then the article needs to be updated, which could be as minor as the removal/replacement of the citation with a reliable one, to rewriting an entire section that was based on flawed premises. If the citation to a retracted paper was intentional, like in the context of a controversy noting that a paper was later retracted, you can replace {{retracted|...}} with {{retracted|...|intentional=yes}}, suppressing the red notice
...Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.[1]
Otherwise? Well... carry on as usual. But if you see one of those big red notices, don't panic. Treat it like any other unreliable source, and update the article accordingly. If a retraction paper (or one with an expression of concern) is intentionally cited, then simply follow the instructions and replace {{retracted|...}} with {{retracted|...|intentional=yes}} (or{{expression of concern|...}} with {{expression of concern|...|intentional=yes}}) to suppress the red notice.
Good idea for a bot - never heard about it or seen it in action, but I'm glad it's back up and running - kudos to those responsible! —Ganesha811 (talk) 12:41, 8 June 2024 (UTC)[reply]
Aren't retracted articles below unreliable sources in terms of quality? With an unreliable source, we don't know if the information is good or not; with a retracted article, we know that somebody has found a major problem. No? —Compassionate727(T·C)00:48, 9 June 2024 (UTC)[reply]
In the early 1990s, it was hypothesized that autism could be caused or aggravated by opioid peptides like casomorphine that are metabolic products of gluten and casein. Based on that hypothesis, diets that eliminate foods containing either gluten or casein, or both, are widely promoted, and many testimonials can be found describing benefits in autism-related symptoms, notably social engagement and verbal skills. Studies supporting those claims had significant flaws, so those data were inadequate to guide treatment recommendations.
The claim in the first sentence—that there was a hypothesis of a link between autism and metabolic products of gluten or casein—is supported by a 1991 paper. That paper isn't online, but the abstract says it is based on a study of 30 children. It doesn't really matter whether it is accurate, because it does establish what it is being cited for—that a hypothesis of the sort existed, not that it is true.
The second sentence is uncited—but it does not seem particularly unbelievable that people jumped on a hypothetical link between gluten/casein metabolisation and autism and promoted diets around it of dubious credibility. Wikipedia has an entire category on autism-related pseudoscience, after all.
The final sentence—saying that there isn't really any truth to the hypothesis—cites a 2006 review article and a now retracted Cochrane review originally published in 2008. Why was the Cochrane review retracted?
This review was withdrawn from the Cochrane Library in Issue 4, 2019, as it has not been updated since its last revision in 2008. The editorial group responsible for this previously published document have withdrawn it from publication.
They're not saying "this is debunked nonsense", they're saying "it's old and unless the article gets updated, our policy is to retract it". If an article is retracted because it's trash, then yes, that's a good reason to not rely on it. If an article is retracted because a diligent body like the Cochrane Collaboration are concerned that it might be in need of an update... that doesn't immediately make it useless. Editors have brains, and it is fine to use them. Carefully considering how Wikipedia uses it as a source and the reasons behind the article's retraction would be strongly advised in deciding whether to continue using it. If the article was retracted because it is a load of made up junk, that's a real problem. But that's not always going to be the case. —Tom Morris (talk) 16:33, 9 June 2024 (UTC)[reply]
It's incredibly bad luck that I happened to randomly choose this as an example then. I'm glad people who actually know something about these topics are thinking hard about it. Between the rapid rise in conspiracy theories, the replication crisis and the preprint-to-tweet pipeline that flourished during the peak years of COVID, knowing stuff seems to have had a tough time recently—any effort to try and fix that that is commendable. —Tom Morris (talk) 14:17, 10 June 2024 (UTC)[reply]
I'd argue it was good luck. I didn't have room in the piece to got into the nitty gritty, and didn't really know how to summarize the issue concisely and intelligibly. But here, with more space, you summarized it better than I ever could. Headbomb {t · c · p · b}23:10, 10 June 2024 (UTC)[reply]
I'm personally just tempted to remove all retracted notices from Cochrane Reviews and let Pi bot deal with it. But we should have that discussion at WT:MED. Headbomb {t · c · p · b}17:04, 9 June 2024 (UTC)[reply]
Mastodon
LinkedIn
Facebook
X (Twitter)
Reddit
+ Add a comment
Discuss this story