Continuing Medical Education: JAMA Opinion Article on CME Bias – Calls for Searching Out Illusionary Bias

In the seventies among the puritanical religious community, it was vogue to play rock
albums backwards looking for subliminal messages.   Some people even made careers out of exposing what they considered the hidden messages in records. Today we are seeing similar puritanical hysterics called for in accredited CME.  Today’s puritans using the guise of opinion papers in prestigious medical journals are calling for physicians to closely look for unapparent bias in CME programs, claiming they are not looking closely enough or perhaps not playing the recordings backwards.

Continuing medical education (CME) is required for physician relicensing, credentialing, and recertification, and physicians often depend on CME to learn about new tests and therapies. Pharmaceutical or medical device manufacturers support some CME programs, which has caused concern to some about the potential for bias and conflicts of interest.

While critics have been quiet over the past few years, a recent opinion article published in the Journal of the American Medical Association (JAMA) makes the conclusion that because “bias in CME can ultimately harm patients, … CME programs should adopt stronger measures to identify and eliminate bias, in addition to policies regarding conflicts of interest.” When we received the email on the article it read “JAMA Online First” we assumed it must be an important research paper, but it was not.

The article is written by industry-critic Bernard Lo, MD, who now works at The Greenwall Foundation, which recently started a “bioethics program.” The paper also received funding and support from the Greenwall Foundation, however, the “views expressed” do “not reflect the view” of the Foundation only the authors personal biases. As background Dr. Lo was the chairman of the Institute of Medicine (IOM) committee on Conflict of Interest who’s report basically called for the end of all commercial support of CME and all other physician-industry collaboration with the exception of some research.

Our main points of rebuttal to his arguments include:

There has been marked decrease in CME courses due to retraction of commercial support 

There is no evidence to support that commercially supported CME lacks value as compared to other CME sources of funding 

Dr. Lo’s proposal to ferret out bias imposes added expense to accomplish something that will add little value to the actual education in the CME activity

Dr. Lo’s proposals are based on previous failure of large research studies to show any bias in commercially supported CME and reveal that Dr. Lo conflates acceptable bias based on facts with unacceptable bias based purely on opinion.

The article provides no balance or attempt at empirical evidence to support its assertions ignoring important characteristics of accredited CME—such as the Accreditation Council for Continuing Medical Education’s (ACCME) accreditation criteria and Standards for Commercial Support—many of which contradict what the authors try to prove.

For example, there is no known evidence that commercially supported CME programs—with or without the author’s so-called “bias”—have directly or indirectly caused patient harm. Further, there is no known evidence that commercially supported CME programs have led to inappropriate prescribing or treatments or medically unnecessary procedures.

To the contrary, we have cited numerous CME programs, including several that have received commercial support, which show that CME has improved patient outcomes in areas such as multiple sclerosis, hypertension, COPD, ICU patients, improved taking of family history by physician assistants, Sepsis, healthcare-associated infections, reduction in CT scans, and several other areas. More recently, accredited CME was shown to reduce healthcare costs.

The authors ignore the firewalls and rules, standards and oversight that have been in place for several years to prevent the alleged bias the authors claim. For example, in 2012, 15% of nationally accredited CME providers were placed on probation and another 30% were required to submit progress reports, demonstrating the ACCME’s clear commitment to compliance. Below is a further summary of Lo’s article and counterarguments to his paper.

Funding: Is There Bias in Commercially Supported CME?

Despite significant declines in commercially supported CME and the increase of “other source” income, Lo attempts to paint a picture that CME is biased by pointing out that in 2011, “75% of accredited CME providers received support from commercial entities.” Yet, Lo fails to give the entire picture of commercial support.

As we noted earlier in August, commercial support for CME in 2012 decreased by 10.3% ($77,659,472) from 2011; marking the sixth year of decline. Commercial support now represents 27.3% of the total CME funding, down from 46% of total funding in 2007.

Additionally, in 2012, “income from other sources” made up 59% of total income, an increase of 6% from 2011. Income from other sources represents income other than commercial support and advertising and exhibits income. Examples of income from other sources include participant registration fees, and allocations from a provider’s parent organization or other internal departments.

While 75% of CME providers may have received commercial support in 2011, Lo fails to explain that 80% of CME providers received $1 million or less in commercial support, with almost half (48%) bringing in $100,000 or less—including 25% of providers bringing in no commercial support.

In 2012, 51% of CME providers received $100,000 or less, including ~27% receiving no commercial support. Moreover, the percentage of CME providers receiving over $100,000 in commercial support decreased in 2012 from 2011—with only 17.5% of providers receiving $1-10$ million and only 1.3% of providers receiving greater than $10 million.

A large portion of nationally accredited CME providers (30.4%) received only $100,000 to $1 million in commercial support—typically enough for a few programs depending on the nature of the CME program. Lo fails to acknowledge these numbers, despite using the same data from the ACCME Annual Report.

Moreover, while Lo believes CME programs receiving commercial support may be “biased,” he fails to acknowledge that the majority of CME activities (82%) did not receive commercial support in 2012, accounting for approximately 81% of physician participants and 78% of non-physician participants.

Thus, only 18% of CME activities received commercial support, and of these activities, only 19% of physician and 22% of non-physician participated. These percentages have remained stable since 2010, the first year the ACCME presented data about commercial support at the activity level.

Consequently, although commercial support of CME, the number of activities receiving such support, and the number of physician and non-physicians partaking in such activities have decreased dramatically, the article asserts that commercial support of CME raises concerns about conflicts of interest (COIs) and bias.

We have written extensively about several large studies showing undiscernible bias in commercially supported CME (Cleveland Clinic; Medscape, and UCSF).  Lo only acknowledges one of these studies—the Medscape study by Ellison. Furthermore, we have cited several surveys demonstrating that physicians find value in industry supported CME.

Nevertheless, Lo expresses concern because to date, “CME oversight has focused more on disclosing and managing COIs, particularly financial relationships between industry (such as drug companies) and CME sponsors or physicians, than on reducing bias during CME presentations.” This may be a literary sleight of hand or perhaps an oversight by Dr. Lo. The ACCME standards for commercial support state “The provider must have implemented a mechanism to identify and resolve all
conflicts of interest prior to the education activity being delivered to learners“. Nowhere does the ACCME mention managing COI’s.

Bias and COIs

Dr. Lo creates and uses his own definition of COI: “situations that present an unacceptable risk [or probability] of undue influence or bias.” Contrary to his interpretation, the ACCME acknowledges a COI when circumstances allow “an individual an opportunity to affect CME content about products or services of a commercial interest with which he/she has a financial relationship.” Financial relationships create actual COIs in CME “when individuals have both a financial relationship with a commercial interest and the opportunity to affect the content of CME about the products or services of that commercial interest.”

Dr. Lo explains that “undue influence occurs when a CME course or presentation is inappropriately influenced by financial or personal considerations.” As an example of undue influence, Lo notes that if a commercial interest (e.g., drug company) “were permitted to suggest or choose speakers, they would likely select physicians who favor their products.” In fact, Lo maintains that even if such a speaker gave an unbiased presentation, “the undue influence in their selection undermines the independence of medical education and public trust in the educational process.”

Unfortunately, it is unclear why Dr. Lo used this example when he acknowledges that, “commercial supporters may no longer suggest or select speakers and topics.” Including this example attempts to portray a practice that has been extinct for almost a decade and the comprehensive ACCME Standards for Commercial Support (SCS) (1.1., 3.2, 5.1 and 5.2) prevent such selection from ever happening. Moreover, the Centers for Medicare & Medicaid Services (CMS) recognized the value of the ACCME SCS in its final rule implementing the Sunshine Act by exempting payments to speakers at ACCME-accredited programs where the commercial interest has no role in the content or selection of speakers as the SCS already require.

Furthermore, Dr. Lo understates what “undue influence” truly means. Courts of law have long recognized that showing “undue influence” is no easy task. To sustain a claim of undue influence, one must show coercive interference with a person’s will of such severity that it causes him to do what he would not otherwise have done, or if not by coercion, then the use of other means that subverts the will causing a choice or judgment that would not otherwise have been made.  Proving such undue influence in the context of commercially supported CME is likely impossible and does a disservice to the necessary and constitutionally protected scientific exchange that is the hallmark of accredited-CME.

First, a CME program or speaker would need to be biased to such an extent that a physician would be deceived. The ACCME SCS and federal oversight makes this highly unlikely. Second, even if the program slipped through the cracks and was “biased,” a physician learner would have to be coerced in some way—either by the speaker, content, or materials—to ignore the decades of training and experience the physician has to wholeheartedly and without question, subvert to the bias of the CME program. Other than hypnotism or subliminal messages, physicians are not likely to make a clinical decision based solely on a CME program without first validating the information and data first, particularly when COI’s and commercial support have been disclosed.

Nevertheless, Dr. Lo believes that “bias in CME is more troubling” because it involves “presenting information, drawing conclusions, or making recommendations that are not scientifically valid or not supported by the weight of rigorous evidence.” He notes that many COIs “do not result in bias,” and that bias may result from “factors other than financial relationships with industry, including intellectual commitment to a therapeutic approach, limited expertise about the topic, methodological shortcomings, and poor judgment.”

Dr. Lo expresses his concern with bias because while disclosure of COIs is an “essential first step in addressing them,” he believes this mechanism is “counterproductive” because speakers or faculty may give “more biased advice after making a disclosure, knowing that listeners have been put on guard and may discount the advice.” Yet the only support he offers for this assertion is a study regarding disclosure of industry funding in a “hypothetical” clinical trial.

The study, published in the New England Journal of Medicine (NEJM), found that industry sponsorship negatively influences physician’s perception of methodologic quality and reduced their willingness to believe and act on trial findings, independently of the trial’s quality. Such a comparison is problematic because it compares apples and oranges; disclosure of industry support in CME is vastly different from clinical trials in several ways.

First, clinical trial data typically focuses only on the drug being tested. Thus, industry support of such trial has a greater likelihood of distrust given the clear financial interest present. In CME programs, ACCME SCS 5.1 requires that the content or format of a CME activity or its related materials “must promote improvements or quality in healthcare and not a specific proprietary business interest of a commercial interest.” Thus, when a CME provider and its faculty or speaker make a disclosure, this standard prevents any level of bias about the commercial interest from being made.

Further, in addition to the ACCME ensuring through its oversight that a CME program does not promote a commercial interest, the FDA, DOJ, and HHS-OIG all have oversight of any CME program that might cross the line from accredited-CME into product promotion. CME providers typically don’t allow “more biased advice after disclosure” because it would risk jeopardizing their accreditation status and bring legal scrutiny.

Second, clinical trial data does not necessarily need to give other information about similar drugs. Conversely, ACCME SCS 5.2 requires that presentations give “a balanced view of therapeutic options,” which may include the use of “generic names” and “trade names from several companies” where available. Thus again, CME providers and programs are prevented from introducing bias even after disclosure because a balanced presentation is required.

Finally, clinical trial data presents typically information on one trial. Physicians rarely attend a CME program just to learn about one clinical trial. Consequently, CME programs present many of the latest clinical data as well as relevant and recent scientific papers, updates from NIH, FDA, other federal health agencies, and other healthcare stakeholders. CME providers and their subject matter experts aggregate thousands of pages of the latest information and data and are engaged in constant and rigorous academic scholarship to ensure that CME programs include not only the latest and most up-to-date information, but a tremendous breadth of such information.

Additionally, while Dr. Lo recognizes that COI disclosure is important because it allows CME directors and learners to assess and manage COIs and potential bias, he fails to acknowledge the vast depth of the mechanisms and firewalls in place CME providers use. For example, ACCME SCS 1.1 clearly explains that a CME provider must ensure that all decisions are “made free of the control of a commercial interest,” including (a) Identification of CME needs; (b) Determination of educational objectives; (c) Selection and presentation of content; (d) Selection of all persons and organizations that will be in a position to control the content of the CME; (e) Selection of educational methods; (f) Evaluation of the activity.

Accreditation Criterion 7-10 also require CME providers to develop activities/educational interventions independent of commercial interests; appropriately manage commercial support, respectively; maintain a separation of promotion from education; and actively promotes improvements in health care and NOT proprietary interests of a commercial interest.

In addition, ACCME SCS 3.2 clearly explains that a CME provider “cannot be required by a commercial interest to accept advice or services concerning teachers, authors, or participants or other education matters, including content, from a commercial interest as conditions of contributing funds or services.”

Managing COIs and Bias

Despite these clear differences, and ignoring the weight of evidence demonstrating non-significant bias in commercially supported CME (as noted in the three studies above), Dr. Lo believes that physicians who attend commercially supported CME programs “probably underestimate bias” for four (4) reasons.

First, Dr. Lo maintains that physician “learners typically have insufficient knowledge about the topic to assess whether presentations are biased.” This statement is entirely incorrect.

Physicians go through almost a decade or more of training and education, which teaches them to question and challenge every scientific or medical piece of evidence until they have the best answer(s) applicable to the patient or case at hand. Included in this training process is teaching physicians how to weigh evidence and potential bias in all shapes and forms. Determining whether bias is present in a CME presentation is no different. Commercial support of CME has been around almost as long as CME and physicians are well aware of the disclosures being made and how to assess the information being presented.

Moreover, the fact that complaints are submitted to the ACCME, individual CME providers, and the FDA all demonstrate physicians have sufficient knowledge to assess bias in CME presentations. If more bias was present, we would expect to see more accredited CME providers on probation, losing their accreditation status, or receiving letters from law enforcement. However, we are unaware of any pervasive existence of such events. Consequently, Lo may believe that the absence of such complaints and actions suggests that physicians have insufficient knowledge to report the bias. Yet, as the three studies above have already demonstrated, the minimal complaints and regulatory actions is a product of a well functioned CME accreditation system that prevents and significantly reduces any potential COIs and bias.

Second, Dr. Lo believes that learners have little incentive to give detailed, thoughtful comments. This statement is untrue and unsupported by any evidence. Those who works in the CME industry or for a CME provider would be able to tell Dr. Lo about the dozens of verbal and written comments or post-activity surveys CME providers receive on a range of topics, including bias. In fact, contrary to Lo’s unfounded assertion, physicians have strong incentives to give detailed and thoughtful comments because they understand that doing so will lead to improved CME programs, assistance in helping CME providers produce outcomes measures, and are often required to earn the full CME credit.

Third, Dr. Lo asserts that questionnaires to detect bias in CME commonly pose only one general question about bias, ask for global, intuitive judgments, and do not define key terms operationally. Typical items are “The data presented in the program were incomplete or framed in a biased fashion” and “The data were presented in an unbalanced manner (italics added).” Respondents to such generally worded items identify bias through implicit “we know it when we see it” judgments—a standard that has been widely criticized. Dr. Lo cites one study, which suggests that asking CME learners to check off specific manifestations of bias leads to a higher rate of reporting.
In other words, learners may need to be told what to look for in order to see it. This method has been proven in polling to produce its own set of “biases” and often leading questions are just that “leading”.

Finally, Dr. Lo argues that even when questions did ask about specific manifestations of bias, they addressed only the simplest forms of bias, such as showing a company logo on slides or mentioning trade names.

The last two assertions by Lo are problematic. In essence, Lo asserts that because current measures for bias in CME are not finding any bias, the questionnaires and questions CME providers are asking must be wrong or physicians do not understand them. Dr. Lo is attempting to generate more fuel for his own scholarship by bashing an otherwise well-run accredited CME system. He cannot find evidence of any bias, fail to acknowledge the evidence showing no bias (three largest studies), and does not acknowledge the ACCME SCS.

Moreover, physicians are constantly assessing the bias of a CME program during the program itself. To suggest that a physician would need individual questions or more tailored questions to each aspect of the program to find evidence of bias amounts to a witch-hunt. Physicians need to pay attention during CME programs to the actual content and data. It is already difficult enough to get physicians to retain knowledge and improve clinical care based on CME programs. Attempting to incorporate more distractions and questions about potential bias will only further inhibit the quality of CME programs and the impact they are having. And for what purpose? To bolster Dr. Lo’s own bias and his ethics career of crying “conflict” when it has been proven no evidence exists.

Consequently, Lo believes that CME providers should use “scoring systems or checklists” to better identify bias in CME, instead of using global or implicit evaluations. He asserts that CME providers should “Ask about specific manifestations of bias that can be readily and consistently identified,” such as:

1. Does the presentation compare options for managing the condition, including generic drugs and lifestyle changes? Are their advantages and disadvantages compared? Framing a presentation as “New drugs for XX” rather than “Treatment of XX in 2013” leads to bias in favor of new drugs. Even if presentations about studies of new drugs are not distorted, bias occurs if data on new drugs are not put in the context of other approaches and established therapies.
   

First, the ACCME SCS already require this in CME programs. As noted above, ACCME SCS 5 requires that the “content or format of a CME activity or its related materials must promote improvements or quality in healthcare and not a specific proprietary business interest of a commercial interest” and presentations “must give a balanced view of therapeutic options,” including use of generics and multiple trade names.

Second, this question suggests something that the article itself did not do: acknowledge the advantages and benefits of commercial support as well as the mechanisms already present to reduce and manage bias in CME.

Additionally, this point is only relevant to a small minority of CME programs sponsored by commercial support. For instance, only certain types of diseases will have generic drugs and lifestyle changes that would make sense from an educational and evidence-based standpoint to include, such as type-one diabetes or various cancers. On the other hand, physicians treating complex and chronic diseases with debilitating side effects are seeking new information and clinical data about drugs that may be in clinical trials because their patients have no other options.

1. Does the presentation use a critical literature review or meta-analysis to summarize the totality of evidence? A comprehensive critical review of evidence or a meta-analysis as the most rigorous way to provide context is far more preferable to speakers’ opinions unsupported by evidence or by evidence-based practice guidelines from a trustworthy source.
   

This suggestion is problematic because many physicians attend CME programs for the specific purpose of learning about the recent developments in medicine, rather than historical overviews of older data or comparisons that may not be as relevant. More importantly, many treatments that may be discussed at CME programs regarding cutting edge treatments are small phase II or emerging phase III trials that are saving lives. How can a CME provider do a meta-analysis for such treatments?

1. Are the limitations of data for new therapies discussed? Evidence from pivotal clinical trials is often limited in many ways—insufficient power to detect important but rare adverse effects and drug interactions; exclusion of patients with advanced age, poor functional status, and comorbidities; and lack of long-term outcomes and postmarketing safety data. Furthermore, adherence to medications in real-world settings may be lower than adherence in clinical trials with research staff. Thus, the results of pivotal trials may not hold for patients commonly observed in clinical practice. Presentations on new drugs should discuss these limitations and compare the findings for new drugs to the much more extensive data regarding established therapies.
   

As already noted, ACCME SCS 5 requires that CME presentations “give a balanced view of therapeutic options.” In addition, many CME providers have begun using case studies and other interactive patient programs in faculty and speakers are able to share their experience with treatments based off of drug interactions, age, and other factors noted above. However, these new approaches can be expensive, which demonstrates continued need for commercial support to off-set the costs to physicians.

1. What important pertinent topics are missing from the presentation or course? Certain topics, such as patient-physician communication, counseling, and quality improvement, are infrequently covered in industry-supported CME, although these topics represent common and important challenges in clinical practice.
   

Given the strict time limitations and already dwindling resources, it is likely difficult for many CME providers to incorporate more topics into accredited CME programs, particularly those discussing new clinical data and treatments. It would almost be impossible for a CME program to include all of the content to meet the educational objectives of the CME program and to achieve educational outcomes on such objectives while also addressing issues such as communication or counseling. While such issues are certainly important, the value of CME to physicians is by interacting with experts and learning about recent changes in the management and treatment of a particular treatment.

Dr. Lo recommends that physicians who are knowledgeable about the topic should review slide decks before presentations using these “checklists.” He notes that reviews should be knowledgeable about evidence-based medicine and should have no promotional relationships with relevant companies. This expertise is needed to assess whether presentations lack balance or omit important information or issues.

Dr. Lo misunderstands that most nationally accredited CME providers already do this, not only to comply with ACCME accreditation and SCS standards, but also to ensure the highest quality education and training. We do agree with Lo’s suggestion that peer reviewers be rewarded through offering CME credits, as some medical journals do for peer reviewers, and including such reviews in promotions and tenure packets.

Ultimately, Dr. Lo concludes that the ACCME should “conduct periodic reviews of how CME programs identify and respond to bias and, in the spirit of quality improvement, disseminate best practices and deidentified reviews to other CME programs.” This recommendation, as we have noted through this article, may not be necessary given the significant oversight ACCME already practices and the level of compliance CME providers engage in with respect to commercial support.

We agree with Dr. Lo “Bias is presenting information, drawing conclusions, or making recommendations that are not scientifically valid or not supported by the weight of rigorous evidence.” Without any evidence of bias in CME or patient harm, Lo’s attack on commercially supported CME, is a self-serving article that is unbalanced and biased. So Dr. Lo where is your rigorous evidence in lashing out at ethical CME providers. Dr. Lo’s bias and unbalanced discussion are impossible to miss. Dr. Lo has turned his “bias” into “ethics” that is quite a slight of hand.

Next time you attend a CME program, perhaps the critics can record it and play it backwards, who knows they may find hidden messages such as “prescribe more …..” or use ….” Or perhaps take the faculty at their word for being honest physicians committing significant personal time in helping physicians understand ever more complex medicine.

For the record, I have my own set of biases, I believe that commercial support of CME is good for society, CME providers have invested heavily in insuring fair balance and resolving conflicts of interest that may exist. The practice of CME has made my life better and the life of countess friends of mine who would have died prematurely without the collaboration that currently exists to improve our health.

Many thanks to the doctors with the Association of Clinical Researchers and Educators who gave us many more arguments than we were able to use in writing this article. The bottom line is they all thought Dr. Lo’s original article was pure “rubbish.”

 “There are no conflicts of interest among rational men in a free society.”  Ayn Rand.