A recent article in PLOS One, titled “Citability of Original Research and Reviews in Journals and Their Sponsored Supplements,” examined the use of “supplements to biomedical journals, which can be used as a forum to highlight a particular disease state or intervention.” The article focused on how such supplements are “often sponsored by a commercial enterprise with a stake in the topic being discussed, such as a pharmaceutical company wishing to promote their medications.” It was noted that “usually the supplement is funded through an unrestricted grant and CME credits may or may not be provided.”
Issues regarding supplements that have already been discussed are the fee charged for a supplement, recommendations regarding the role of the journal editor, selection of a supplement editor, disclosure of funding source, the role of the funding organization, individual journal policies and commentaries in editorials. This list however, does not include how supplements themselves can be cited in scholarly works, which can be a concern for some who believe that “the contents of supplements are perceived to be biased in favor of the sponsor.”
To address this controversy, the article contrasted citation rates for articles appearing in the 2000 and 2005 issues of the Journal of Clinical Psychiatry (JCP) and its supplements, using citation profiles in the ISI Web of Science database. Using this journal was a good choice, since it has 35,613 subscribers and is the 3rd most cited psychiatric journal in the world.
The articles surveyed included “a mixture of research reports, including reports of randomized controlled trials, and reviews.” Papers that are submitted to JCP “undergo the usual and customary peer review process by expert consultants, i.e., typically two or more independent reviewers are charged with examining the quality of the study or review and its potential importance to the field.”
The article noted however, that “supplements to JCP undergo a different review process including a planning session (telephone or live) at which the authors have the opportunity to comment on each other's presentations/submissions, a prepublication review by the pre-designated Chair of the activity for accuracy and fair balance (the Chair often writing an introduction as well as one or more articles in the supplement), and a prepublication review for fair balance by a reviewer from the CME Institute of Physicians Postgraduate Press, Inc. The role of the journal editor in the production of the supplement is not explicitly stated.”
Since the results of the study showed that “articles published in supplements in JCP are often cited when compared to the parent journal,” the author discussed the potential implications of such findings. Before making recommendations however, the author noted that “not all supplements from all journals are financed by pharmaceutical or medical device manufacturers.” In addition, “some journals may already have in place a rigorous peer-review process for their supplements.” Due to these factors, the author recommends that the study “should be replicated using another journal which makes much use of supplements in a field of specialty other than psychiatry.”
The article acknowledges that “citation counts by themselves may not be necessarily reflective of the impact an article has on actual day-to-day clinical practice, and that the number of citations an article receives may not be representative of the number of readers who have accessed the article.”
Interestingly, the citation rate of supplements does suggest that highly cited articles, regardless of where they were published, “may contain information that other authors may have found useful to support their arguments.” This information may also be cited “to identify methodological shortcomings and to show how the citing study intends to rectify those limitations.” Clearly then, supplements serve a valuable purpose.
Although the study found supplements useful, the research was limited because “the individual quality of each cited report was not assessed, and no distinction was made between reports of randomized controlled trials, other types of original reports, and reviews.” The study also did not “further scrutinize funding sources or potential biases in the articles themselves,” and did not “explore whether articles published in supplements are preferentially cited by other articles that have been sponsored in some way.” The study also left “unaddressed “the issue of ghost authorship and ghost management of supplements.”
Conclusions
Journal supplements play an important role in translating science into clinically useful educational programs. Providing context for clinical studies and often answer questions that practicing clinicians encounter. There is a rigorous scientific review that goes into them and being cited suggests the importance of this translation work.
Perhaps “articles published in JCP supplements are robustly cited,” applying additional guidelines to monitor such supplements may be overly burdensome to cash strapped journals that already have peer-review in place, and include supplements, which are not commercially supported. As a result, recommendations to review supplements must be extremely careful because their impact is influential in guiding clinical and research practice, as well as shaping critical thinking.