Cleveland Clinic Study of CME Bias Found No Evidence of Difference Between Supported and Non Supported Programs
In a recent edition of the Journal of Academic Medicine, Steven Kawczak and William Carey of the Cleveland Clinic Foundation discussed “The Effect of Industry Support on Participants’ Perceptions of Bias in Continuing Medical Education.” The purpose of the study was to obtain prospective evidence of whether industry support of continuing medical education (CME) affects perceptions of commercial bias in CME activities. The article noted that there was no funding or financial support, to conduct the study and there were no other disclosures.
All of the data used was from the Accreditation Council for Continuing Medical Education (ACCME) Annual Report Data 2007.
In order to carry out their research, the authors analyzed information from the CME activity database (346 CME activities of numerous types; 95,429 participants in 2007) of a large, multispecialty academic medical center—the Cleveland Clinic—to determine whether a relationship existed among the degree of perceived bias, the type of CME activity, and the presence or absence of commercial support. The study looked at activities since 2004—the same year in which the ACCME updated its Standards for Commercial Support. The response rate for participants completing the evaluation was approximately 70% for all activities.
Findings
In response to the question: Overall, was this activity satisfactorily free from commercial bias?”
– 97.3% to 99.2% (mean: 98.4%) of participants answered “yes.”
In responding on a four-point scale to the request, “Please rate the degree to which this activity met the ACCME requirement that CME activities must be free of commercial bias for or against a specific product:”
– 95.8% to 99.3% (mean: 97.2%) of participants answered “excellent” or “good.”
When analyzed by type of funding relative to commercial support, activities were deemed to be free of commercial bias by:
– 98% for no commercial support (149 activities);
– 98.5% for single source commercial support (79 activities); and
– 98.3% for multiple source commercial support: (118 activities)
When asked to rate the degree to which the CME activity met the ACCME requirement that CME activities must be free of commercial bias for or against a specific product:
– A range of 0% to 0.7% of respondents (depending on the type of activity) gave a response of “poor,” the lowest rating on the four-point scale; and
– Between 1.0% and 6.8% of respondents (again, depending on the type of activity) assigned the two lowest scores (“fair” or “poor”) to the CME activities in which they participated.
In other words, over 90% of respondents thought that the CME activity did a good or excellent job making the CME activity free of commercial bias for or against a specific product. In fact, participants judged activities with no commercial funding being relatively free of commercial bias the same as they judged those activities with funding. Moreover, “activities that might be considered to be at greatest risk of bias—those with single sources of commercial funding— actually had the smallest proportion (0.1%) of “poor” ratings.”
Discussion
In 2007, ACCME reported that support of CME providers by industry reached $1.2 billion, or more than 47% of total revenue to CME providers offering education certified for American Medical Association (AMA) Physician’s Recognition Award Category 1 credits. This funding represents an extremely important role for doctors because “the alliance between industry and medicine, when properly managed, benefits both the education of physicians and their care of patients.” Unfortunately, some feel that these alliances, and the associated benefits of industry support for CME, are problematic. For example:
– “The U.S. Senate Finance Committee recently concluded that ACCME guidelines may be insufficient to prevent commercial bias within CME;” and
– The Macy Foundation recently issued a report claiming that the presence of commercial support in continuing education risks distortion of content may compromise evidence-based decision making, upholds a mindset among health professionals that CME should be paid for by others, and even impedes the adoption of effective modes of learning. The participants in the Macy Foundation’s effort felt that even a “firewall,” regardless of its strength, could not properly mitigate bias and that a new structure for the funding of CME should be sought.
One attempt to end industry support of CME through the AMA House of Delegates “was voted on and returned to the CEJA (for a third time) for reconsideration and specifically requested further consideration of the implications of such an action on all stakeholders.” The report “lacked contemporaneous data on CME activities with industry funding.”
While critics may think that “the integrity of professional medical education may be sacrificed from this alliance, and that any industry support corrupts content,” to date, “no published studies have addressed the relationship between commercial support and perceived bias in accredited CME activities.”
Although some research has explored the changes in physicians’ prescribing behavior that are related to commercial funding, “limited attention has been given to the impact of commercially funded CME on prescribing behavior and on physicians’ opinions about bias.” In fact, today, “no evidence exists for a link between commercial funding and issues of bias that may result from such funding.”
With such a gap in research and evidence, the Association of American Medical College’s (AAMC) belief “that medicine itself can, and should, effectively regulate all real or perceived conflicts of interest,” seems more realistic. In fact, critics who are worried about CME content created by industry or bias towards developing CME content that is of interest to commercial supporters is misplaced.
Since 2004, ACCME’s updated Standards for Commercial support require accredited CME providers to show “that everyone who is in a position to control the content of an education activity has disclosed all relevant financial relationships with any commercial interest to the provider” (Element 2.1) and that there is “a mechanism to identify and resolve all conflicts of interest prior to the education activity being delivered to learners” (Element 2.3). CME providers often augment these external standards with additional guidelines and requirements. Specifically, many CME providers believe that content should be “based on independent research identifying educational gaps in physician performance or issues in patient care.”
Conclusion
From the results listed above, the study found no evidence that commercial support results in perceived bias in CME activities, and bias level was quite low for all types of CME activities and was not significantly higher when commercial support was present. Also, participants actually felt that single source funding were more free from of bias. The study also found that across a wide range of types of CME activities, no difference in commercial bias was apparent between those programs with and without commercial support. In fact, participants even detected some bias in activities with no funding.
The authors suggested that although similar data are needed from other types of CME providers, it is reasonable to conclude that the perception of commercial bias does not have a strong correlation to the presence of commercial funding.
Ultimately, while the goal of industry is commercial in nature, the authors found that “such a goal is not inimical to the goal of unbiased, high-quality medical education.” As a result, this data shows that a CME provider with suitable oversight to ensure compliance with ACCME Standards for Commercial Support can ensure that commercial funding does not affect commercial bias in an activity.
This data also shows that industry support of CME does not undermine the integrity of professional education, and therefore, the prohibition of commercial support for CME is not needed.