The American Academy of Family Physicians (AAFP) Division of Continuing Medical Education (DCME) recently announced that it is “moving to optimize its CME faculty selection process, in part by offering easier access to online tools for faculty,” according an announcement from DCME Medical Director Ann Karty, M.D. During the past few months, Karty explained, the Academy has :
- Made renovations to the CME faculty section of the AAFP website;
- Updated resources, including a new CME Faculty Resource Center; and
- Disseminated e-mail blasts, mailers and satisfaction surveys — all designed to enrich the overall faculty portfolio.
“AAFP CME faculty members share their passion to teach and learn with their family medicine colleagues,” Karty said. “The recent updates to the recruiting and selection process components are designed to streamline the overall faculty selection process.”
The Academy’s online CME Faculty Resource Center now offers several different content development tools, including a webcast hosted by Karty on designing an effective case presentation or case studies. The site also includes links to guidelines for writing tests and learning objectives, as well as a resource on incorporating performance measures into CME activities. Below is a description and summary of some of these new resources in more detail. Finally, AAFP has also added links to a number of AAFP-recommended, evidence-based CME resources — both free and subscription-based — round out the complement of tools.
In addition to these resources, the Academy also provides a link(www.nfeinitiative.org) to online training that outlines the differences between certified CME and promotional activities through the National Faculty Education Initiative. AAFP CME faculty are required to complete this training.
The Academy offers opportunities for authors, editors and content development faculty to serve as live, audio, video or enduring presentation faculty members.
Guidelines for Writing Learning Objectives
AAFP states that the goal of a performance-based learning objective is to describe the behavior in measurable terms that you, a faculty member, would expect to observe of the learner upon completion of this learning activity. Then, the resource provides several Do’s and Don’ts. For example:
- Do: Describe the observable action that you would expect to see the learner “doing” upon completion of the learning activity.
- Don’t: Do not describe the instruction that you, the faculty member, will perform in order to teach the learner.
- Do: Use measurable terms to describe the actions of the learner.
- Don’t: Do not use unmeasurable terms such as understand, know, be familiar with, comprehend, learn, or appreciate.
- Do: Describe only one action in each objective.
- Don’t: Combine more than one action using “and.”
- Do: Write learning objectives that are supported by the content of the learning activity.
- Don’t: Do not write a learning objective based on content that is insufficiently addressed.
The resource then gives very granular recommendations, tips and examples of how to word learning examples, showing poor wording (and explaining why) and then showing “better wording.” There is also an extensive “Verb Selection Guide” on the second page.
Guidelines for Writing Tests
This document explains that the goal of multiple-choice tests is to allow the learner to demonstrate competence in applying concepts described by the learning objectives from the associated CME activity. Following are tips, along with examples, for writing quality multiple-choice tests:
Tests should:
- Relate directly to instructional learning objectives
- Consist of one question/answer/explanation per learning objective per direction of AAFP project manager
- Contain a case study, when appropriate, with multiple questions related to the case.
- Avoid testing knowledge that is controversial, especially when it is incomplete or debated
- Avoid testing knowledge of medical trivia
- Contain questions/answer choices/explanations that are the same for both the pre-test and post-test; re-arrangethe order for the post-test
Questions should:
- Include only relevant information
- Contain as much of the question information as possible such that phrasing is not repeated in each answer choice(See example #3)
- Ask for the correct, not the “wrong”, answer – i.e. “Which of the following is NOT…”.
- Avoid absolute terms, such as always, never, all, or none
- Avoid imprecise terms, such as seldom, rarely, occasionally, sometimes, few, or many; instead, use absolute numbers (e.g. 15%, rather than few)
- Avoid cues, such as may, could, or can
- Spell out words for which you use an acronym, or abbreviation; follow with the acronym
Answer choices should:
- Contain incorrect answer choices (distractors) that are accurate but do not fully meet the requirements of thequestion; options should never be implausible, trivial, or nonsensical
- Follow grammatically from the question, and not stand out as a result of phrasing
- Relate to each other in kind (e.g., all diagnoses, all tests, all treatments)
- Be similar in grammar, length, and complexity, with incorrect answers (distractors) being plausible but clearlyincorrect
- Not include none of the above and all the above
- Be listed in order (e.g. numeric, chronologic)
- Be independent of each other and should not overlap
- Be varied in the positioning of the correct answer; i.e., not all correct answers should be ‘A’
- Four to five options is optimal
Explanations should: Concisely but thoroughly explain why the specific answer option is correct. The explanation does not need toexplain why the incorrect answer options are not correct. The document then gives examples of questions and best practices for tests.
Incorporating Performance Measures
This document gives a number of resources for evidence-based measures, including
- AMA-convened Physician Consortium for Performance Improvement (PCPI) measures
- NCQA (National Committee for Quality Assurance): HEDIS® measures
- PQRI (Physician Quality Reporting Initiative)
- AAFP resource page on performance measurement
- AAFP resource page on practice management tools; and
- The National Quality Forum (NQF) provides measures that can link process and outcome. See NQF-Endorsed® Standards
It also provides examples from a recent CME activity to show some of the outcomes based measures.