The Institute of Medicine (IOM) issued two reports last week “that it said will provide an objective and consistent framework for clinical practice guidelines, and standardize systematic reviews of comparative effectiveness”—also known as comparative effectiveness research (CER). The studies were requested by Congress and sponsored by the Department of Health and Human Services (HHS).
The first report, entitled Clinical Practice Guidelines We Can Trust, includes eight standards and recommendations.
IOM’s second report, “Finding What Works in Health Care: Standards for Systematic Reviews,” addresses the “myriad and often competing guidelines for clinical recommendations by offering what it said are 21 standards to ensure objective, transparent, and scientifically valid reviews.”
Background
According to IOM’s press release, “clinical practice guidelines and systematic reviews of the evidence base for health care services are supposed to offer health care providers, patients, and organizations authoritative guidance on the comparable pros and cons of various care options, but too often they are of uncertain or poor quality. There are no universally accepted standards for developing systematic reviews and clinical practice guidelines, leading to variability in the handling of conflicts of interest, appraisals of evidence, and the rigor of the evaluations.”
IOM Clinical Practice Guidelines
1. Establishing transparency: The processes by which a clinical practice guideline (CPG) is developed and funded should be detailed explicitly and publicly accessible.
2. Management of conflict of interest (COI): Prior to selection of the Guideline Development Group (GDG), individuals being considered for membership should declare all interests and activities potentially resulting in COI with development group activity, by written disclosure to those convening the GDG.
Disclosure should reflect all current and planned commercial (including services from which a clinician derives a substantial proportion of income), non-commercial, intellectual, institutional, and patient/public activities pertinent to the potential scope of the CPG. Disclosure of COIs within GDG:
- All COI of each GDG member should be reported and discussed by the prospective development group prior to the onset of their work.
- Each panel member should explain how their COI could influence the CPG development process or specific recommendations.
IOM also recommend that members of the GDG should divest themselves of financial investments they or their family members have in, and not participate in marketing activities or advisory boards of, entities whose interests could be affected by CPG recommendations. Additionally, IOM noted that:
- Whenever possible GDG members should not have COI.
- In some circumstances, a GDG may not be able to perform its work without members who have COIs, such as relevant clinical specialists who receive a substantial portion of their incomes from services pertinent to the CPG.
- Members with COIs should represent not more than a minority of the GDG.
- The chair or co-chairs should not be a person(s) with COI.
- Funders should have no role in CPG development.
3. Guideline development group composition: The GDG should be multidisciplinary and balanced, comprising a variety of methodological experts and clinicians, and populations expected to be affected by the CPG. Patient and public involvement should be facilitated by including (at least at the time of clinical question formulation and draft CPG review) a current or former patient and a patient advocate or patient/consumer organization representative in the GDG.
4. Clinical practice guideline–systematic review intersection: CPG developers should use systematic reviews that meet standards set by the IOM’s Committee on Standards for Systematic Reviews of Comparative Effectiveness Research. When systematic reviews are conducted specifically to inform particular guidelines, the GDG and systematic review team should interact regarding the scope, approach, and output of both processes.
5. Establishing evidence foundations for and rating strength of recommendations: For each recommendation, the following should be provided:
- An explanation of the reasoning underlying the recommendation, including:
- A clear description of potential benefits and harms.
- A summary of relevant available evidence(and evidentiary gaps), description of the quality (including applicability), quantity (including completeness), and consistency of the aggregate available evidence.
- An explanation of the part played by values, opinion, theory, and clinical experience in deriving the recommendation.
- A rating of the level of confidence in (certainty regarding) the evidence underpinning the recommendation.
- A rating of the strength of the recommendation in light of the preceding bullets.
- A description and explanation of any differences of opinion regarding the recommendation
6. Articulation of recommendations: Recommendations should be articulated in a standardized form detailing precisely what the recommended action is and under what circumstances it should be performed. Strong recommendations should be worded so that compliance with the recommendation(s) can be evaluated.
7. External review:
- External reviewers should comprise a full spectrum of relevant stakeholders, including scientific and clinical experts, organizations (e.g., health care, specialty societies), agencies (e.g., federal government), patients, and representatives of the public.
- The authorship of external reviews submitted by individuals and/or organizations should be kept confidential unless that protection has been waived by the reviewer(s).
- The GDG should consider all external reviewer comments and keep a written record of the rationale for modifying or not modifying a CPG in response to reviewers’ comments.
- A draft of the CPG at the external review stage or immediately following it (i.e., prior to the final draft) should be made available to the general public for comment. Reasonable notice of impending publication should be provided to interested public stakeholders
8. Updating:
- The CPG publication date, date of pertinent systematic evidence review, and proposed date for future CPG review should be documented in the CPG.
- Literature should be monitored regularly following CPG publication to identify the emergence of new, potentially relevant evidence and to evaluate the continued validity of the CPG.
- CPGs should be updated when new evidence suggests the need for modification of clinically important recommendations. For example, a CPG should be updated if new evidence shows that a recommended intervention causes previously unknown substantial harm, that a new intervention is significantly superior to a previously recommended intervention from an efficacy or harms perspective, or that a recommendation can be applied to new populations.
IOM Standards for Systematic Reviews
In the Medicare Improvement for Patients and Providers Act of 2008, Congress directed the IOM to develop standards for conducting systematic reviews. The recommendations from the second report emphasized similar standards and principles including:
- Manage bias and conflict of interest (COI) of the team conducting the systematic review; and
- Manage bias and COI for individuals providing input into the systematic review
Discussion
After announcing the two IOM reports, Sheldon Greenfield, MD, executive director, Health Policy Research Institute, University of California, Irvine, and chair of the IOM committee on guidelines, asserted that, “These standards are necessary given that there is little documentation to judge the quality and reliability of many of the existing clinical practice guidelines.”
Dr. Greenfield emphasized that the “Practice guidelines provide valuable data and guidance that not only inform individual decisions about care but ultimately could also improve overall healthcare quality and outcomes.”
Alfred O. Berg, MD, professor of family medicine, University of Washington School of Medicine, Seattle, and chair of the committee that wrote the report on systematic reviews, noted that, “this report presents the ‘gold standard’ to which those who conduct systematic reviews should aspire to achieve the most reliable and useful products.”
Dr. Berg recommended that the standards in the IOM reports be adopted to “minimize the chances that important health decisions are based on information that may be biased or erroneous.” He did however recognize that doing so, will “take an investment of resources and time to achieve such high standards.”
Commentary
The problem with this report and others like it is the underlining belief system that working with industry to develop and commercialize new therapies is somehow inherently bad and should be managed or eliminated.
The problem with this philosophy is that it fails to recognize expertise and value. In all other industries working with industry by academicians is considered valuable experience. But in medicine the ivory tower has decided that they can determine clinical guidelines alone without the input of those who design and work on randomized controlled clinical trials, the backbone of industry-academia collaboration.
The problem with this is that it is the equivalent of setting a rule that NASCAR drivers should not have any relationships with industry therefore lack the expertise of the latest and best science and drive slower cars but they would be pure.
No patient and I literally mean no patient wants a physician who only has knowledge of old therapies setting “guidelines” for other physicians. These recommendations if adopted will put us one step closer to “government controlled” healthcare.
Rather than spending all our energy determining eligibility based on relationships with product developers. Would not our time and resources be better spent in finding cures for diseases and coming up with guidelines that implement those therapies? We have had guidelines for 50+ years and almost all have been out of date within weeks of their release.