Study Finds High-Impact Medical Journals Guilty of Anti-Industry Bias

Over the past few years, this website has been dedicated to highlighting the importance of ethical, physician-industry relationships that advance science and medicine and improve patient care and clinical outcomes. 

Economists have concluded that extensive interaction between academic researchers and practicing physicians with industry has facilitated the development and dissemination of biomedical diagnostics, devices and therapies.  Despite the significant improvements in healthcare, such as increased lifespan, decreases in death rates for cancer and cardiovascular disease, and better management of diabetes and hypertension, critics persist.  

A small, but vocal group, has consistently attacked these relationships, expressing their concern for “bias” and potential “conflicts of interest;” calling for everything from a complete ban, to increased transparency, which has culminated with the passage of the Physician Payment Sunshine Act.  These self-professed “ethicists” and “conflict of interest specialists” have made a career out of attacking physician industry collaboration—including some individuals who previously worked for industry.  They have sold and written books, given speeches, and even dedicated blogs to attacking such relationships.  

This has led to academic health centers, states and the federal government to institute regulations designed to monitor, limit or eliminate such interactions based on concerns that such relationships may degrade the performance and reporting of biomedical research and also induce physicians to behave in a manner inconsistent with cost-effective or ethical patient care—which are loosely defined under the operational term ‘financial conflicts of interest’ (COIs). 

While we are strong supporters of increased transparency, we have consistently recognized that giving the public information about how much a company pays a physician and using payment categories (i.e. consulting, education) is insufficient and does not provide the context needed to understand the complex nature of physician-industry collaboration.  

A recently released study highlighted the tension regarding the proper role physician-industry relationships play in advancing science and medicine and improving patient outcomes.  The study presents compelling findings about the bias nature of high-tier medical journals concerning these relationships.  Specifically, the authors conclude that “Prestigious” medical journals overwhelmingly make claims that industry research and marketing are corrupt and therefore collaboration corrupts doctors.  The media reports these articles and arguably, this influences policy.  The paper documents this fact and shows that the claims in journals are not backed by evidence and that balanced interpretation is lacking. 

The study, written by Roman Lesko, Samuel Scott, and Thomas P Stossel, MD, was supported in part by grants from the Association of Clinical Research Organizations (ACRO), The Association of Clinical Researchers and Educators (ACRE) and from the Searle Freedom Trust.  Prior to this publication, four medical journals rejected the paper, either subjecting it to impassioned negative reviews or refusing to review it due to “lack of interest.”  Nature Biotechnology journal published it despite such reviews. 

Study 

The authors sought not only to analyze whether the positive and negative aspects of industry-academic relationships were equally represented in top-tier medical journals but also to assess the weight of evidence in the COI literature that patient outcomes or public attitudes are indeed negatively affected by corporate interactions with academics and physicians. 

They analyzed papers published in four journals selected on the basis of their high (>20) impact factors: The Journal of the American Medical Association (JAMA), The Lancet, Lancet Neurology and The New England Journal of Medicine (NEJM).  They identified articles by performing keyword searches on the websites of the selected journals and on PubMed.  The 108 articles selected for analysis encompassed reports of original research, reviews, editorials and other commentaries. 

Publications on this topic began to appear in the 1980s and peaked around 2000.  The selected articles were published from the early 1980s to 2008.  Earlier papers primarily concerned research relationships.  Later papers addressed diverse industry-related activities, such as company marketing to physicians, corporate subsidy of physician education and other activities, use of professional writers of industry-sponsored publications or paying physicians to provide advice to investors in medical technologies.

Three readers analyzed the articles by posing four questions: 

1. Does the article emphasize benefits or risks of relationships, or is it neutral?
2. Does the article provide evidence, and, if so, is the evidence quantitative, anecdotal or both?
3. Does the article specifically draw conclusions regarding patient care outcomes or public concerns about relationships, usually described as ‘public trust’?
4. Does the article recognize and, if so, address, points of view contradicting its conclusions? 

Examples of quantitative evidence in the relationship-to-industry papers included compilations of the prevalence of all types of physician-industry or academic-industry relationships, effects of industry marketing on physician prescribing behavior, the influence of industry relationships on voting patterns of US Food and Drug Administration (FDA) panelists and results of surveys of physicians, patients or officials of academic health centers concerning attitudes regarding physicianindustry relationships. If an article presented evidence by citation of another article to buttress its conclusions, the conclusion was considered evidence-based.   

The authors determined whether an article addressed ideas contrary to its conclusions, noting whether the article critically analyzed opposing opinions, simply mentioned but did not attempt to rebut a different viewpoint or did not recognize a view other than its own.  Citing a less extreme opinion but one still on the side of the argument advocated by the article did not qualify as addressing balance. 

The preponderance of articles (n = 96, or 89% of the sample) unambiguously emphasized their risks.  In many cases, the risk emphasis was obvious from the paper’s title (e.g., “Just how tainted has medicine become?” or “Impugning the integrity of medical science: the adverse effects of industry influence”). These articles either did not discuss or only briefly mentioned benefits of physician-industry or academic-industry relationships, and all of them recommended more stringent regulation of such relationships 

For example, one commentary published in Lancet asserted that there is “grave danger because of the nature of its dealings with the pharmaceutical industry, the medical profession is forfeiting public confidence.”  The author, however, knew of “no firm data from opinion polls that would allow [him] to substantiate his hypothesis.”  Instead, he was “sufficiently impressed by comments from … the public, politicians … and the media.”  

Another research article concluded that the “medical profession needs to develop a more effective policy on conflict of interest” after looking at cardiologists’ connections with industry.  The products of the companies with which the cardiologists consulted, however, bore no consistent relationship to the consultants’ opinions.  In addition, a widely cited commentary used to justify institutional conflict of interest policies makes a wide-ranging claim that the most challenging and extensive conflicts of interest “emanate from relationships between physicians and pharmaceutical companies.”  The paper, however, provides no evidence to support this statement, which makes severe quantitative assertions (e.g. “regularly”).  

Findings 

Of these articles, 17% contained original data and were classified as research papers or else as ‘special articles’ by the journals in which they were published.  The rest were editorials or commentaries.  These two types of articles were analyzed separately. 

Whereas, by definition, all the risk-emphasizing research articles contained evidence, fewer than half of the risk-emphasizing commentaries did.  Half of the risk-emphasizing research articles and over two-thirds of the non-research articles included a statement to the effect that relationships to industry negatively affected patient care outcomes or caused public distress.  Of the total set of risk-emphasizing articles, nearly two-thirds contained no mention of an opposing point of view and only 5% critically analyzed it. 

Twelve articles either emphasized benefits of physician-industry relationships (9) or presented a neutral, balanced assessment (3).  All of the benefit-emphasizing articles presented some form of evidence to support their arguments and discussed an opposing viewpoint in detail.  The differences between the risk-emphasizing commentary articles and the benefit-emphasizing or neutral articles were statistically significant on key measures assessed in this study, including presentation of evidence, assumptions concerning patient outcomes and addressing an alternative point of view. 

Taking the above results into consideration, it is clear that the preponderance of articles published in the four highest-impact medical journals that publish primary research focused on problems concerning COI relationships. 

These articles differed qualitatively from benefit-emphasizing academic-industry relationship papers.  Most risk-emphasizing articles presented no evidence and many of the ones that did present evidence extrapolated that it had a bearing on patient outcomes or public attitudes.  This finding was “intriguing” to the authors because they were “unaware that such an effect ha[d] been established.”  On the contrary, three risk-emphasizing papers that attempted to identify adverse outcomes of relationships found little evidence for such consequences. 

Most risk-emphasizing articles made no mention of dissent from their risk-emphasizing views or opportunity costs of industry relationship regulation. In contrast, all the publications emphasizing benefits of relationships also explored the case made for risks of relationships.  The authors noted that, “the dominance of risk-emphasizing papers and the low prevalence of opposing viewpoints in those papers have contributed to the evolution of policies concerning academia-industry relationships.” 

Conclusion 

In an ideal scenario, policymakers faced with incomplete information engage in vigorous debate to deliberate different viewpoints.  Undermining this ideal is a phenomenon defined by behavioral economists as a ‘conformity cascade’.  In a conformity cascade, policy emerges not from an objective weighing of alternatives but from social pressure to conform and from lack of dissent.  

When an issue arises abruptly as a perceived problem, as exemplified by the flood of COI publications in the 1980s, and policymakers do not deeply study it but rather imitate opinions of influential others, especially in the absence of opposing arguments, the tendency is for them to take ever more extreme positions.   

This analysis reveals that the discussion of academic-industry relationships in top-tier medical journals has been unbalanced.  The authors wondered whether this imbalance has in turn negatively influenced policymaking, resulting in more restrictive policies than the evidence supports, particularly in terms of how COIs relate to patient outcomes or public attitudes.  It also points to the need for editors of top-tier journals to be open to a broader and more inclusive diversity of voices when considering articles on academic-industry relationships. 

Association of Clinical Researchers and Educators Press Release