Добавил:
Sekretar
kiopkiopkiop18@yandex.ru
t.me/Prokururor I Вовсе не секретарь, но почту проверяю
Опубликованный материал нарушает ваши авторские права? Сообщите нам.
Вуз:
Предмет:
Файл:Ординатура / Хирургия / Библиотека им академика М.И. Перельмана / Книга_2725_Библиотеки_им_академика_М_И_Перельмана.pdf
X
- •Preface
- •References
- •Contents
- •Author Biographies
- •About the Series Editors
- •Family Medicine Advocacy Summit
- •Robert Graham Center Workforce Studies
- •HealthLandscape
- •Residency Resources
- •Medical Education Policy Development
- •Residency Program Solutions
- •Residency Leadership Summit
- •Chief Resident Leadership Development Program
- •Educational Resources
- •Continuing Medical Education (CME) Tracking
- •Graduate Medical Education Awards
- •Summary
- •References
- •AAFP Structure
- •AAFP Governance
- •Workforce Mission
- •AAFP Workforce Advocacy
- •Residency Redesign
- •References
- •Osteopathic Recognition Requirements
- •Osteopathic Recognition Program Essential Personnel
- •Osteopathic Faculty
- •Osteopathic Recognition Resident Requirements
- •Osteopathic Educational Program Core Competencies
- •Osteopathic Program Evaluation
- •References
- •Reference
- •5: Association of Family Medicine Administration and the Key Role of the Residency Program Coordinator
- •Appendix 1
- •References
- •References
- •Membership Categories within NAPCRG
- •The AAFP Robert Graham Center
- •AAFP National Research Network (NRN)
- •Family Physicians Inquiry Network (FPIN)
- •The Family Medicine Physician-Scientist Pathway (PSP) Program
- •NAPCRG Grant Generating Project
- •Building Research Capacity (BRC)
- •Conclusion
- •References
- •Reference
- •9: Family Medicine Residency Accreditation
- •Why Accreditation Matters
- •ACGME Organizational Structure
- •The Family Medicine Review Committee (RCFM)
- •The Clinical Learning Environment Review (CLER)
- •The Site Visit
- •Initial Accreditation Period
- •Subspecialty Fellowship Application
- •The Annual Program Review
- •The Next Accreditation System Annual Review Process
- •Accreditation Data System (ADS)
- •Citations
- •Accreditation Status
- •Program Requirement Development
- •Competency-Based Medical Education (CBME)
- •References
- •10: Graduate Medical Education Funding
- •Introduction
- •Residency Operating Costs
- •Medicare GME Funding
- •Residency Caps
- •Rural Track Programs
- •Inpatient Prospective Payment System (IPPS) Subtype Hospitals
- •Rural Referral Centers
- •Sole Community Hospitals
- •Medicare-Dependent Hospital
- •Medicaid GME Funding
- •Teaching Health Center Funding
- •Veteran’s Administration GME
- •Children’s Hospital GME (CHGME)
- •State Line Item Funding
- •Hospital Supplemental Funding
- •Patient Care Revenue
- •Philanthropy
- •Conclusions
- •References
- •Introduction
- •Informed Self-Assessment
- •Faculty Development
- •Summary
- •References
- •12: Core Competencies, Milestones, and Entrustable Professional Activities
- •Entrustable Professional Activities
- •Entrustment
- •Pearls
- •Pitfalls
- •References
- •Objectives
- •Educational Strategies
- •Implementation
- •13: A Practical Approach to Curriculum Development
- •Introduction
- •Cognitive Learning Theory
- •Behavioral Learning Theory
- •Humanistic Learning Theory
- •Goals
- •Conclusion
- •References
- •Introduction
- •Preparation Phase
- •Information Technology Systems
- •Medical Student Rotations
- •Interview Season
- •Application Screening
- •Interview Day
- •Second Looks
- •Postinterview Communication
- •Ranking
- •Using R3®
- •Match® Week
- •SOAP®
- •Conclusion
- •References
- •Preboarding
- •Planning New Resident Integration
- •Baseline Assessment
- •Individual Learning Plan
- •Integration Components
- •Longitudinal Integration
- •Learner/Residency/Institutional Goal Alignment
- •Conclusion
- •References
- •Introduction
- •History
- •Residency Application
- •Conclusion
- •References
- •References
- •Introduction
- •Identifying Performance Issues
- •Medical Knowledge
- •Patient Care
- •Interpersonal Skills/Communication
- •Professionalism
- •Systems-Based Practice
- •Remediation
- •Medical Knowledge
- •Patient Care
- •Interpersonal Skills/Communication
- •Professionalism
- •Systems-Based Practice
- •Conclusions
- •References
- •19: Faculty Recruitment: Best Practices
- •Background
- •Seeking Out Suitable Candidates: Recruitment Methods
- •Unique Challenges: Faculty Who Are New Residency Graduates
- •Concluding Thoughts: Always Recruit!
- •References
- •20: Faculty Performance
- •Faculty Performance Concerns
- •Appendix 1: Sample Family Medicine Residency Program Director Job Description
- •Job Summary
- •Educational Delivery
- •Evaluation
- •Administration
- •Appendix 2: Sample Family Medicine Residency Core Faculty Job Description
- •Job Summary
- •References
- •21: Faculty Development
- •Introduction
- •Clinical Skills
- •Scholarly Work
- •Tasks that Family Medicine Residency Faculty Perform
- •Role Modeling
- •Faculty Peer Support
- •Teaching
- •Training Courses
- •Workshops
- •Learning by Doing
- •Self-Directed Learning
- •Career Development
- •Organizational Resources
- •References
- •Introduction
- •Promoting Loving Accountability
- •Clinical Setting
- •Curriculum
- •Social Connection
- •Assessment
- •Vision Forward
- •Handling Microaggressions
- •Financial Investment
- •References
- •Building Trust
- •Positionality
- •Set the Stage
- •Normalize Not Knowing
- •Probing, Not Prodding
- •Accurate
- •Addressing Implicit Bias
- •Micro-Level
- •Macro-Level
- •Conclusion
- •References
- •Introduction
- •Background
- •Risk Factors
- •Stigma
- •FSMB Recommendations Regarding Licensing Questions
- •Physician Health Programs (PHP)
- •Individual-Level Actions
- •Conclusion
- •References
- •References
- •The Clinical Learning Environment
- •Feedback
- •References
- •27: Teaching Maternal Health Care
- •ACGME Requirements
- •Obstetric Family Medicine Faculty Shortage
- •Maternity Care Curricular Elements
- •Preconception Care
- •Prenatal Care
- •Immunizations
- •Common Conditions Which Complicate Pregnancy
- •Breastfeeding
- •Intimate Partner Violence (IPV)
- •Substance Use
- •Food Insecurity
- •Inpatient Obstetrical Care
- •References
- •Introduction
- •Family Medicine Residency Advocacy Curricular Development
- •Evaluation
- •Conclusion
- •References
- •Faculty Leadership
- •Resident Leadership
- •Patient Satisfaction
- •Professional Conduct
- •Ethical Considerations
- •Curriculum Development
- •The Hidden Curriculum
- •Conclusion
- •Context Matters
- •References
- •30: Teaching Behavioral Science
- •Introduction
- •Core Mental Health Topics
- •Patient-Centered Communication Skills
- •Contextual Care
- •Integrated Behavioral Health Care
- •Master Adaptive Learner
- •Clinic-Based Learning
- •Classroom-Based Learning
- •Interprofessional Education
- •Balint Groups
- •Conclusion
- •Appendix 1: Sample Behavioral Health Rotation (Myerholtz 2023)
- •References
- •31: Teaching Evidence-Based Medicine
- •Introduction
- •Longitudinal Deep Dives
- •Journal Club
- •Family Physicians Inquiries Network
- •Case-Based Learning
- •Day-to-Day Integration
- •Curriculum Development Tips Summary
- •Institutional Involvement
- •Summary
- •References
- •32: Teaching Practice Management
- •Background
- •Current Curriculum
- •Activities
- •Quality Improvement Project
- •Virtual Practice Instructions
- •Presenters
- •Topics Covered
- •References
- •Overview
- •Why Training Is Needed
- •Stressing Communication
- •Balint Group
- •Putting It All Together
- •References
- •Introduction
- •Broad Scope
- •Primary Contact
- •Emerging Technology
- •Rural Generalism
- •Public Health Advocate
- •Rural Practice Systems
- •References
- •Introduction
- •Basic Logistics
- •Operational Considerations
- •Billing Requirements
- •Additional Funding Considerations
- •Engaged Leadership
- •The FMC-Residency Partnership
- •Scheduling
- •Data-Driven Improvement
- •Empanelment
- •Panel Weighting
- •Panel Transitions
- •Team-Based Care
- •Population Management
- •References
- •Introduction
- •Personnel Management
- •Meeting Management
- •Communication Management
- •References
- •37: Managing Change in Family Medicine Residency Programs
- •Introduction
- •What Is Change Management?
- •Change Management Case Study
- •Kotter Model
- •Step 1: Create Urgency
- •Step 4: Communicate Buy-In
- •WIFM Worksheet
- •Step 5: Empower Others
- •Step 6: Generate Quick Wins
- •Step 7: Consolidate Gains
- •Conclusion
- •References
- •Negotiating Styles
- •Collaboration
- •Summary
- •References
- •Anticipating Task Cycles
- •Residency
- •Home Life
- •Seize Control
- •Delegate
- •Delegate Residency Chores
- •Delegate Domestic Chores
- •Exercise
- •Lunch
- •Organizing
- •Summary
- •References
- •Introduction
- •References
- •Introduction
- •Anticipate Common Disasters
- •Learn About Your Regional Emergency Infrastructure
- •Learn Your Organization’s Command Center Structures
- •Here Are Some Alternative Care Site Scenarios That May Arise
- •Regional Emergency Infrastructure
- •Managing Your Program During Disaster
- •Community Recovery
- •Consolidate Your Learning
- •Author Background
- •References
- •Conclusion

88
D. Keister and V. Brohm
that are characteristic of CBME.These include focusing on
outcomes, emphasizing abilities, de-emphasizing time-based
training, and promoting greater learner-centeredness
(Table11.2). Education that follows these principles ensures
that every graduate is prepared for practice.
Expanding on these key concepts, Van Melle convened a
group of experts using a Delphi method to create a CBME
Core Components Framework to describe the ideal implementation for CBME. In traditional medical education, the program structure is the primary focus with the learning,
assessment, and outcomes seen as necessary products of an
ideal structure. The Van Melle framework centers clearly articulated outcome competencies that are described as a sequenced
progression in a triangle that both create and are inuenced by
Table 11.2 Key principles that are characteristic of CBME. (From Frank etal. [26])
Main principle Elaboration
Focusing on outcomes
In an era of greater public accountability, medical curricula must
ensure that all graduates are competent in all essential domains
Emphasizing abilities
Medical curricula must emphasize the abilities to be acquired There is too much emphasis on knowledge and not enough on skills,
De-emphasizing time-based training
Medical education can shift from a focus on the time a learner spends
on an educational unit to a focus on the learning actually attained
Promoting greater learner-centeredness
Medical education can promote greater learner engagement in training A curriculum of competencies provides clear goals for learners
three other factors: (1) Programmatic assessment that “sup-
port and document the developmental acquisition of competencies,” (2) Tailored learning experiences that “facilitate the
developmental acquisition of competencies,” and (3)
Competency-focused instruction that “promote(s) the developmental acquisition of skills.” Van Melle describes how each
component might appear in practice, the principle behind the
components, and the conceptual frameworks from educational
theory that support them in an excellent resource for readers
who care to investigate this topic in more detail (2019).
The ultimate goal of CBME is to assure the development
of competent physicians who are master adaptive learners
[16] who will maintain their expertise throughout their
careers.
Not all current curricula explicitly dene desired outcomes
Not all current curricula address all of the desired outcomes
Not all current curricula assess or ensure that graduates have acquired
all of the necessary abilities
In the health professions, assessment scores should not be
compensatory from one domain to another (i.e., excellent knowledge
does not compensate for poor communication skills)
Medical education needs to be transparent for learners, teachers, and
the public with respect to its goals and effectiveness
Standards must be criterion-oriented
Medical education tends to emphasize process issues (e.g.,
instructional methods) over outcomes (e.g., graduate performance and
satisfaction)
Medical education must prepare trainees for practice
Content that does not contribute to preparation for practice should be
dropped
attitudes, and their synthesis into observable competencies
An emphasis on the abilities of learners should be derived from the
needs of those served by graduates (i.e., societal needs)
Educational objectives as an organizing framework should be replaced
with a hierarchy of competencies
Time is a resource to be tailored to the needs of teachers and learners
Current curricula and credentialing tend to emphasize xed times
spent in training
Learners may progress at different rates and may achieve threshold
competencies faster or slower than the average peer
Greater emphasis should be placed on the developmental progression
of abilities and on measures of performance
Greater exibility may make some curricula more efcient and
engaging
A roadmap of milestones provides a transparent path to achieve the
competencies
An individual learner can adjust their own learning using the
milestones

11 Designing Assessment toMeet theChallenge ofCompetency-Based Medical Education
Laying theFoundation forCBME
The International Competency-Based Medical Education
(ICBME) Collaborators created the fundamental principles
of CBME at their 2013 summit. These three principles
include: education must be based on the health needs of the
populations served; the primary focus of education and
training should be the desired outcomes for learners rather
than the structure and process of the educational system;
the formation of a physician should be seamless across the
continuum of education, training, and practice [13]. The
shift to CBME utilizing these principles will ensure learners are assessed on their application of medical knowledge
in alignment with their local population and system needs,
which vary widely within countries and across the globe
[13]. For there to be seamless transitions in the advancement of learners, all entities including UGME, GME, and
board certifying bodies need to share a common image of a
suitable physician, starting with the “end in mind” [51, 59].
Carraccio et al. describe the commitments required for
medical educators to successfully implement CBME,
including the commitments to: Teaching, assessing, and
role modeling the broad range of identied competencies;
Supervision that balances patient safety with the professional development of learners; Transparency with all
stakeholders; Commitment to the empowerment of learners; The effectiveness and efciency of assessment strategies and tools; Basing transition decisions on competence
rather than time; Advancing CBME through workplace
assessment, program evaluation, and research; Faculty
development; and Collaboration [13].
Utilizing CBME requires medical educators to empower
learners to play an active role in their education. These
commitments correlate with key concepts from the Master
Adaptive Learning (MAL) model, which promotes the residents’ ability to recognize and address knowledge gaps
through transparent and specic feedback from all stakeholders in resident education, including self-evaluation [15,
16]. The MAL model (Fig.11.1) contains four gears which
are essential activities for the learner: planning phase, learning phase, assessing phase, and adjusting phase. During the
planning phase, learners identify gaps in their knowledge/
skills/ attitudes, prioritize opportunities for learning, create a
goal, and search for resources. Throughout the learning
phase, residents employ active learning strategies and critically appraise resources. In the assessment phase, the residents implement what was learned or “try it out.” Next, they
compare their unguided self-assessment with an informed
self-assessment utilizing external feedback such as feedback
from faculty, patients, staff, colleagues, and patient outcomes
[16]. Lastly, in the adjustment phase, the learner incorporates
what was learned into their practice.
Fig. 11.1 The master adaptive learner model. (From Cutrer etal. [15])
The gears on this model are powered by the coaching of
medical educators (depicted as a rheostat) combined with the
internal learner characteristic “batteries” of curiosity, motivation, mindset, and resilience [16]. These batteries depict a
learner’s growth mindset through their desire for knowledge
and willingness to engage despite any setbacks or obstacles
that may occur. Although these characteristics are internal,
they can be augmented or suppressed by any number of factors in the learner’s life or in the overarching learning environment. One challenge for FM residents is that their learning
environment is constantly changing (e.g., outpatient clinic,
hospital rotations, off-service rotations, lecture hall, virtual
space, etc.). A key component in MAL is for the learning
environment to be safe. The ACGME Clinical Learning
Environment Review (CLER) Program ensures appropriate
environments by providing sponsoring institutions feedback
on patient safety, health care quality, teaming, supervision,
well-being, and professionalism.
The active participation in learning described in the MAL
model is critical to CBME [30]. The classic model of medi-
89

90
D. Keister and V. Brohm
cal education curriculum design and assessment is a hierarchical creation of curriculum implemented upon learners
with minimal variation to address individual learner needs.
In contrast, CBME requires the co-production of the curriculum and assessment between learners and instructors. The
informed self-assessment required for learners to identify
and plan their learning “requires that learners proactively
seek out assessment from faculty and members of the healthcare team, perform aspects of their own assessment such as
clinical performance reviews or evidence-based practice and
actively engage their assessment data for professional development as part of their own accountability.” Clinicians with
this type of engagement in continual learning are essential to
the improvement in healthcare quality that CBME
promises.
Recommended steps in the planning of the CBME curriculum will be discussed in the remainder of this chapter.
They include: (1) Identifying the abilities needed from grad-
uates; (2) Explicitly dening the required competency and
their components; (3) Dening milestones for the competencies along a developmental path; (4) Selecting educational
activities, experiences, and instructional methods; (5)
Selecting assessment tools to measure progress along the
milestones; and (6) Designing an outcomes evaluation of the
program [26].
Clarifying Competencies, Milestones, EPAs,
andCore Outcomes
Given the new terminology necessary for the implementation of CBME, educators can get confused about the difference between competencies, sub-competencies, milestones,
EPAs and core outcomes. The ACGME Core Competencies,
patient care; medical knowledge; practice-based learning
and improvement; professionalism; interpersonal skills and
communication; and systems-based practice are general
characteristics of a learner that imply knowledge, skills and
attitudes that a learner in any medical specialty must have to
be successful in practice [10, 70]. Each discipline has parsed
the core competencies into sub-competencies that describe
the critical components of practice for that discipline (Family
med milestones). For example, within the patient care competency in family medicine, the rst sub-competency is
“Patient care 1: Care of the acutely ill patient” (Fig.11.2).
Other patient care sub-competencies in family medicine
include care for patients with chronic illnesses, health promotion, care of patients with undifferentiated signs and systems, and procedural care. Within each sub-competency,
each discipline has dened a series of milestones, which are
observable descriptions of the knowledge, skills, attitudes,
and other attributes of elements of a sub-competency orga-
Fig. 11.2 ACGME milestone sub-competency of patient care #1. (From Family Medicine Milestones Workgroup [20])

11 Designing Assessment toMeet theChallenge ofCompetency-Based Medical Education
91
nized in a developmental framework (Family med milestones). The sequenced progression of sub-competency
acquisition within the milestones describes a common path
to growth in learners of every discipline. Indeed, most children do crawl before they walk. However, outliers exist in
normal development. Therefore, learners might acquire more
advanced skills prior to demonstrating basic skills. Similarly,
a normal learning curve might include a delay in a given subcompetency. The milestones detail a common path to skill
acquisition. They are not a perfect description of the learning
journey for every student.
In contrast, Entrustable Professional Activities (EPAs) are
descriptions of discrete tasks or responsibilities that faculty
empower a trainee to perform without supervision when they
have demonstrated the necessary skill [23]. Achieving a level
of ability to be trusted with an EPAs generally requires the
demonstration of multiple milestones within several subcompetencies [71]. For example, one EPA described for family medicine is “Manage inpatient care, discharge planning,
and transitions of care.” To be entrusted with this activity, a
resident must demonstrate skill within the care of acutely ill
patients, a sub-competency of patient care, as well as subcompetencies within medical knowledge, systems-based
practice, professionalism, and interpersonal communication
skills. Thus, as faculty members determine that they trust
residents to perform a given EPA, achievement of milestones
within the family medicine sub-competency grids can reasonably be documented.
Core outcomes can be understood in the context of CBME
as being equivalent to EPAs. The ABFM uses “the term ‘core
outcomes’ because ABFM research (in 2022) showed that
only approximately 40% of family medicine program directors reported they are using the term Entrustable Professional
Activities (EPAs), and what they mean by EPAs varies
greatly [51].” The term core outcome was felt to be more
intuitively clear, and the ABFM hoped that the term would
avoid confusion due to the multiple denitions of EPA
already in use by program directors. However, the ABFM
description of their core outcomes mirrors the denition of
EPAs described within this chapter [51, 52].
Assessment: TheKey toCBME
Ultimately, the success of CBME lies in high-quality assessment. Accurate and timely feedback allows learners the formative information that encourages them to grow and gives
the leaders the educational program the information they
need to provide the learner with the appropriate resources to
advance their training. Although most GME programs have
begun to implement such assessments, there will need to be
substantial growth in the quality of assessments to deliver on
the potential of CBME.
The theoretical model used most often in considering
skill acquisition in medical learners was developed by
Hubert and Stuart Dreyfus [10]. The model details ve
stages of development: novice, advanced beginner, competent, procient, and expert. The stages were originally used
to consider the entire journey of medical education with a
novice being considered a starting medical student and an
expert being a physician in practice. More recently, the
ACGME has found benets in applying the model to the
stages of resident growth with a novice equating to milestone level 1 and an expert to milestone level 5 [18].
Regardless of the scope of the journey included within the
stages, the Dreyfus model allows educators a framework to
describe the development of skills with Novice learners
thinking analytically with little skill in prioritizing information; Advanced Beginners having the ability to apply
analytic and nonanalytic thinking for common problems;
Competent learners being able to see the big picture but
falling back to analytic thinking for complex problems;
Procient learners developing the ability to extrapolate
knowledge to novel situations and live with ambiguity, and
Experts having skill with subtle variations even in complex
problems [18].
Because the ACGME milestones are intended to offer a
formative assessment of resident growth along the Dreyfus
developmental trajectory, it is hoped that they might allow
educators to identify a resident who is behind their peers.
Holmboe etal. [33] investigated the use of the milestones to
facilitate this process in residents from family medicine,
internal medicine, and emergency medicine. Assuming the
goal that graduating residents should achieve milestone level
4 (correlating to a Dreyfus level of procient) in all sub-competencies, Holmboe looked to nd a predictive level to suggest that a resident was unlikely to reach that goal. In their
analysis, they found that residents who had not reached level
3 by the end of PGY2 were less likely to reach the goal of
milestone level 4. However, the predictive probabilities
across the 24 FM sub-competencies ranged from 32% to
67%. Similar ranges of predictive probability existed for the
other specialties. Although the milestones have promise for
this level of predictability, we clearly need to rene this process, either through further revision of the milestones or
through improvement into the assessments that contribute to
milestones assignment.
The ACGME Assessment Guidebook provides an excellent resource for competency-based assessment [32]. The
Guidebook includes a review of commonly used assessment
techniques, describing the pros and cons of each. It also
includes suggestions for the best assessment methods to use

92
D. Keister and V. Brohm
for each ACGME competency. It is unclear how familiar FM
residency program leaders are with the Guidebook or the
degree to which it is used in developing assessment systems
in FM residencies.
Building aSystem ofAssessment
A robust system of assessments across multiple settings is
imperative for sufcient data and feedback in CBME.It is
important to make mindful, intentional choices around the
Fig. 11.3 States of
assessment. (From Norcini
etal. [53])
combination of methods utilized for assessment [76].
Traditionally “too many assessment tools presumably
designed for a competency-based approach are overly reductionist (e.g., a tick box exercise) and there has been a general
lack of a programmatic approach to assessment” [30].
Figure11.3 shows the various states of assessment, revealing
the best coverage of content evaluation in a system of
assessment.
The ACGME assessment guidebook has recommendations for the necessary systems of assessment noted in
Fig.11.4.
Fig. 11.4 ACGME
recommendations for systems
of assessment. (From
Holmboe and Iobst [32])

11 Designing Assessment toMeet theChallenge ofCompetency-Based Medical Education
93
Educators must commit to a routine review of assessment
strategies and faculty development of resident assessment to
collect the most valid competency data. The goal is for all
graduating family medicine residents to reach a level 4 on all
milestones as this correlates with prociency in the Dreyfus
model and readiness for unsupervised practice [14]. A variety of assessment tools are discussed in detail in the ACGME
Assessment Guidebook and descriptions of each tool, including the competencies assessed, validity, feasibility, acceptability, and catalytic effect can be reviewed. Examples of
assessments for each core competency are listed in
Table11.3.
The creation of perfect assessments to evaluate everything a resident should know/demonstrate by the time of
graduation is unrealistic [34]. In a 2023 Webinar discussion
of the implementation of competency-based ABFM board
eligibility Warren Newton stated, “Remember that the perfect is the enemy of the good. Simple, practical, scalable are
priorities” [52].
It is important to recognize the following assessment
pearls for CBME: All assessments are samples; The higher
the stakes, the more samples are needed; Assess what is
important, not just what is easy; All assessment involves
judgment; Quantitative and qualitative methods complement
one another; No single assessment tool can capture all
aspects of clinical competence; Feedback is an essential ele-
ment of assessment; Assessment drives learning; Validity is
the most important characteristic of assessment data; and
Perfect assessment is an illusion [34].
Low-stake assessments have limited consequences to the
learner regarding promotion, whereas high-stake assessments have signicant impact on advancement (Van Der
Vleuten [77] t for purpose). While low-stake assessments
require a lean number of samples, a high-stake assessment
should have multiple assessments from diverse samples.
Quantitative assessments can be considered straight forward
and easily standardized but should be used in a complimentary way to qualitative methods. All assessment involves
judgment, which is why it is imperative for regular faculty
development. ACGME Core Competencies such as
Professionalism or Interpersonal and Communication skills
may feel difcult to assess, but necessary. When learners recognize that all core competencies are equally important
aspects of becoming a physician through assessment, it will
drive their learning to become procient [34, 77].
The Role oftheClinical Competency
Committee
The ACGME requires that all training programs have a
Clinical Competency Committee (CCC) with the purpose
Table 11.3 Assessments for each ACGME core competency. (From Holmboe and Iobst [32])
Competency Competency-based assessment options
Medical knowledge In-training exam
Faculty work-based assessments
Chart stimulated recall, assessment of reasoning tool, and others
Patient care and procedural skills Work-based clinical assessment through direct observation of the
individual during care delivery
Faculty and peer assessment
Standardized assessments
Simulation
Professionalism Informed self-assessment
Multisource feedback, such as a 360-degree evaluation
Patient experience surveys
Interpersonal and communication skills Patient-reported feedback and experience surveys
Multisource feedback, such as a 360-degree evaluation, especially
regarding interprofessional care
Practice-based learning and improvement Evaluation of knowledge, skills, and attitudes from participation in
systematic efforts to improve the quality, safety, or value of health care
services
Audit and feedback of the medical record
Review of medical errors and patient safety events
Evidence-based practice logs
Systems-based practice Feedback from multiple faculty evaluations regarding ability to
practice in a complex health care system
Multisource feedback, such as a 360-degree evaluation, especially
regarding interprofessional care
Assessment of cost-conscious care

94
Fig. 11.5 The assessment
system. (From Andolsek etal.
[5])
D. Keister and V. Brohm
of creating a shared understanding of resident performance
expectations and assessment [2]. A CCC must be appointed
by the program director and, at a minimum, include three
members of the program faculty, at least one of whom is a
core faculty member (ACGME). Additional members must
be faculty or other health professionals who have extensive
contact and experience with the program’s residents
(ACGME). According to ACGME, the literature suggests
that a group of ve to seven diverse individuals is ideal;
therefore, larger programs may need multiple
CCCs.Depicted in Fig.11.5 is The Assessment System in
which the CCC is depicted as the Data Synthesis
Committee. The CCC must review all resident qualitative
and quantitative assessments at least semi-annually to
determine milestone progress and convey individualized
educational recommendations to the program director and
residents. In doing so, the CCC plays an integral role in
CBME by ensuring that the appropriate combination of
evaluation tools is utilized in competency determinations,
while also identifying gaps in the curriculum and/or faculty development regarding feedback. All twice- yearly
residency milestones are submitted to the ACGME
ReviewCommittee for review on the individual and program levels.
Implementation ofCompetency-Based
Assessment
Currently, medical education defaults to a block-based model
where progression is dependent on completion. Implementing
CBME will require a more exible approach to reect the
individualized learning curves of residents. Additionally, moving to a longitudinal curriculum/rotation schedule will allow
teaching continuity between trainees and faculty, enhancing
coaching relationships and allowing for more robust feedback
on progression [56]. Instead of competence being a static view
based on time, Frank etal. [26] propose modiers that specify
domains of ability, context, and stage of practice (Fig.11.6).
Timing of assessments should reect learning trajectories
as knowledge, skills, and behaviors evolve over time [53].
Utilizing a system such as EPAs follows the development of
competence while allowing learners to move at their own
pace [53].

11 Designing Assessment toMeet theChallenge ofCompetency-Based Medical Education
Fig. 11.6 Proposed denitions of CBME and related terms by the ICBME collaborators. (From Frank etal. [26])
95
Informed Self-Assessment
Epstein et al. [19] acknowledge that the “The power of
self- assessment lies in two major domains—the integration of high-quality external and internal data to assess
current performance and promote future learning, and the
capacity for ongoing self-monitoring during everyday
clinical practice.” It is imperative that learner’s informed
self-assessment is fed by clear, timely, specic, constructive feedback, preferably offered by trusted, credible
supervisors in a safe environment [63]. Additionally, a
time for critical and deliberate reection on these assessments is a key activity [63], as external feedback that is
inconsistent with self-assessment may be discounted.
This is because one’s performance is integrally linked to
one’s sense of self [63]. Guidance and facilitation will
enhance this process of reection and the R2C2 Facilitated
Feedback Model can be used to do so through building
rapport and relationship, exploring reactions to performance data, exploring understanding of the content, and
coaching for performance change. The ARCH model can
also be used to provide effective feedback to learners:
Allow/Ask for self- assessment, Reinforce what is being
done well (Knowledge, Skills, and Attitude), Conrm
what needs correction or improvement, and Help the
learner with an action plant for improvement and coach as
needed [9].
Faculty Development
The success of CBME requires that learners receive accurate
feedback. Therefore, it is important that faculty are trained
properly and supported to deliver high-quality formative and
summative assessments (Nousiainen). Validity is the most
important characteristic of assessment data [34] and
characterizes the extent to which a measurement is sound/
correct. The validity of non-standardized assessment resides
in the users rather than the instrument [77]. Because the use
of non-standardized assessment is needed to evaluate
Dreyfus prociency and expertise levels, it is imperative that
investment in faculty education occurs. This is particularly
important for community faculty members who are used in
many FM residencies, but who have minimal formal training
in teaching or assessment skills [64].
The Critical Question: What Happens When
aLearner Does Not Meet Core Outcomes?
The most vexing question for residency programs is what to
do with a learner who fails to meet the competency-based
requirements. The answer is obvious: if a residency program
director (PD) cannot attest that a resident meets the ABFM
core outcomes, the learner should not graduate from FM
residency [51]. However, the ramications of this decision

96
D. Keister and V. Brohm
are massive for both the learner and the program. Despite the
difculties, PDs must have the courage to uphold the social
contract that is inherent in their position as the nal arbiters
of which physicians are eligible for board certication [59].
Fortunately, the development of robust systems of assessment will make this task easier. Certainly, it would be unjust
for a PD to choose not to graduate a resident without any
prior discussion of shortfalls in a core outcome. Thus, the
system of assessment with clear developmental markers is
critical. The goal of CBME is that any learner who is falling
short of a core outcome would be identied early and assisted
in creating a program of study to help them to succeed. This
remediation may require additional time in residency or an
alteration from the residency’s usual curriculum, but most
learners will succeed in meeting core outcomes with this
support [8, 28].
Evaluating Systems ofAssessment
In order to meet the promise of CBME, GME programs must
include a process to evaluate their systems of assessment
with a goal of continuous quality improvement [36, 51]. It is
logical to begin with simple, practical solutions in the short
term, and it will be critical to rene such measures as new
evidence emerges, both internal within and external from
training programs. There are several critical factors that need
to be considered when evaluating systems of assessment.
Most importantly, we must recall that residency programs
are complex systems with many interdependent parts [10,
32]. An evaluation of any component of a GME program
must consider this fact. Bowe and Armstrong [11] offer a
holistic framework for such evaluation, which is simple and
elegant.
At level 1, individual components of the assessment system are evaluated formatively on a frequent basis. For example, an assessment form for an inpatient family medicine
rotation is reviewed by learners, faculty advisors, and the
residency director as it is completed and shared. Residency
management software makes such reviews simple, automatically delivering completed forms to designated individuals.
If a residency director nds that the form offers minimal differentiation between learners, and that few narrative comments are included, they speak informally to the chief
residents and the head of the FM inpatient service about the
issue. The form is adapted based on feedback from these key
stakeholders and all stakeholders monitor future forms to
assess the impact of the change. This example highlights several key components of level 1 evaluation. In this example,
the director identies a problem based on their observation
of their regular workow, though the problem could easily
have been reported at any time by the chief residents, FM
service leader, and inpatient faculty. In all these scenarios,
systems that are already in place lead to the identication of
a problem. The problem is addressed by key stakeholders in
partnership with program leaders. A change is made based
on feedback from stakeholders, and the change is monitored
for effectiveness. This process of continuous quality improvement is familiar to most physicians from the use of the plando- study-act (PDSA) method for clinical improvement [60].
At level 2, the system of assessment itself is monitored for
problems. Through formal and informal processes, educational leaders evaluate the system as a whole. Bowe states,
“comprehensive analyses on combinations of assessment
data are planned and interpreted to identify patterns of component interrelationships, trends in learning trajectories, root
causes of unexpected outcomes, and the predictive value of
various assessment approaches and criteria” [11]. While this
process sounds formal and complex, the reality is it can be
quite simple. Residencies must seek feedback from residents
and faculty about the quality of core learning experiences
and from residents on faculty members’ performance. Such
assessments should be reviewed by the residency’s leadership team to analyze trends. In addition, the review of this
evaluation data is a core function of the Program Evaluation
Committee (PEC). For example, a residency program director (PD) may receive feedback about a PG2 cardiology experience that had previously been well received. The rst sign
might be poor assessment of resident performance, and this
might be followed by resident feedback that the learning
experience is overly challenging. Following these trends, the
PD would bring the concerns to the PEC, and residents might
report that a core cardiology faculty member who had previously taken responsibility for didactics on the rotation has
been out on an unexpected leave of absence. Another member
of the PEC might realize that a recent change to the residency’s formal teaching schedule moved an ECG-reading lecture previously given in the PG1 year to PG2. While the
specic mechanisms that programs use to detect this type of
systemic problem that relates to multiple parts of the residency may be different, the idea that a group is reviewing the
trends and looking into concerning ndings must be a part of
every program. It is important to note the value of both qualitative and quantitative data in this process. Because the
PEC’s work is in practice improvement, the data analysis
need not be at the level of publishable research. Rather, level
2 evaluation needs to allow exibility, rapid detection, and
the ability to respond quickly to problems, as would be typical during PDSA cycles in practice-based quality
improvement.
Finally, level 3 “evaluations occur at periodic intervals to
comprehensively review longitudinal system performance
and determine system readiness to prepare learners for evolving changes in the medical education training continuum and

11 Designing Assessment toMeet theChallenge ofCompetency-Based Medical Education
97
health care system.” The ACGME’s NAS included a requirement for a 10-yr Self-Study that was an excellent example of
level 3 analysis [47, 49, 58, 62]. However, the ACGME
logistically was unable to continue site visits associated with
these Self-Studies [1]. The future of ACGME-mandated
Self-Studies remains unclear. Regardless, level 3 processes
remain an important step for the success of CBME.Therefore,
the steps of the ACGME Self-Study will serve as an example
of a level 3 process. Philibert and Lieh-Lai [58] recommend
that an evaluation group, which could be the program’s PEC
or another subset of stakeholders within the residency,
assembles and aggregates data from Annual Program
Evaluations (APEs). The APE analysis can simply be a
review of the documents themselves with an eye toward
trends that inform a longitudinal assessment of the program’s
improvement efforts. Using the initial analysis as background, the evaluation group meets with stakeholders within
and external to the residency to consider the program’s aims.
Once the aims are reviewed and updated (as necessary), the
evaluation team performs a SWOT analysis, identifying
strengths, weaknesses, opportunities, and threats for the program [22]. The evaluation team then presents the ndings of
the Self-Study with the program leadership and constituents
to come to consensus on improvement priorities for the next
several years. The process is completed by documenting the
ndings and recommendations of the study. As an example,
the PEC for a residency program could schedule an extra
meeting to review the APEs from the last several years. This
could be followed by a resident-faculty meeting to review
the identied trends and discuss the aims of the program. A
SWOT analysis could be conducted by the residents and faculty at that meeting. External stakeholders such as the DIO,
leaders of key off-service rotations, and operational leaders
at the FMP could review the SWOT analysis asynchronously
and offer additional suggestions. The PEC chair could create
a document listing all the ndings to date and share it with
the program leadership, working together to draft a list of
key improvement goals for the next several years. These
goals could be shared with the PEC at its next meeting.
Following a discussion, the APE for the next year could
include an action plan reecting those goals. This process
could occur over a year and would only require a few hours
of time from the PEC, the PD, and a resident-faculty meeting. However, the power of such a process to anticipate
future changes and prepare the program to seize opportunities and respond to threats allows programs to excel, even
amidst uncertain times.
One additional factor that must be mentioned when discussing the evaluation and evolution of GME program
assessment is the difculty of implementation of novel ideas
within complex systems. Given the focus of the ACGME and
ABFM on the implementation of CBME, we can expect the
emergence of evidence about successful assessments processes and systems over the coming years. It is tempting for
a residency leader to think, “I wish they would just tell me
what assessment program to implement and how,” and
equally enticing to imagine that others will do the work of
nding the ideal system that all programs can implement.
However, even if such an ideal system existed, the implementation and optimization of the assessment system within
an individual program would require an understanding of
these principles and a process for evaluation and continuous
improvement. This is true because complex adaptive systems
can create drastically different outcomes even when the same
process is implemented in systems that appear similar [32].
Therefore, the implementation of CBME will require all
GME programs to implement systems of assessment that are
unique to their environment. This caution is not to say that
GME leaders should not look externally for ideas for
improvement. Rather, we must expunge the fallacy that a
process that has been successfully implemented elsewhere
can be implemented locally without change or monitoring.
CBME andTime Variable GME
Traditional medical education is centered around time spent
in a variety of clinical environments [26], usually in blockbased rotations [55]. However, learners come to training
with varying levels of competency and acquire skills at variable rates [26]. One appeal of CBME is an invitation to consider the possibility that learners could complete GME
programs in differing amounts of time [55]. All experienced
GME faculty members have experienced residents who
required more than the required duration of training to meet
the program’s requirements. This may occur for a variety of
reasons, and most residents who require additional time are
able to successfully complete training [8, 28]. CBME offers
promise to detect resident dyscompetency or incompetency
earlier and support remediation by giving specic targets for
improvement [55]. In addition, several pilots demonstrate
the possibility that CBME might allow learners to shorten
their time in training with robust curricula and assessment
systems [48, 56]. Family medicine has begun the journey
into time variable GME in partnership with the ACGME in
the family medicine Advancing Innovation in Residency
Education (AIRE) program that allows longer training and
facilitates innovation in residency curricula [50]. This pilot
began in 2023 and will continue for an uncertain duration as
it assesses possibilities for the future of our discipline.
Соседние файлы в папке Библиотека им академика М.И. Перельмана
