Добавил:
Sekretar
kiopkiopkiop18@yandex.ru
t.me/Prokururor I Вовсе не секретарь, но почту проверяю
Опубликованный материал нарушает ваши авторские права? Сообщите нам.
Вуз:
Предмет:
Файл:Ординатура / Хирургия / Библиотека им академика М.И. Перельмана / Книга_2725_Библиотеки_им_академика_М_И_Перельмана.pdf
98
D. Keister and V. Brohm
Examples ofCompetency-Based Assessment
inFM Residency Education
Unfortunately, the theory supporting competency-based
assessment can be overwhelming, especially for residency
faculty who have limited time and resources [50]. Therefore,
it is important to follow theory with ideas and examples of
successful implementation within GME programs. In the
introduction to their core outcomes, the ABFM provides
concrete examples of assessments appropriate for the ve
core outcomes required for attestation in 2024 [51]. Most
programs will nd that the example assessments listed in
Table 11.4 reect processes that they are already doing.
Remembering the advice to keep the implementation simple
and practical, programs can use existing assessments to attest
graduates’ completion of the core outcomes where possible.
Some minor adjustments or additions may be necessary.
Programs should prepare to modify these existing assessments in response to feedback after the initial year of
attestations.
Some residencies might wish to use their milestone
assessments as markers for attaining core outcomes. For
example, the core outcome “Model professionalism and be
trustworthy for patients, peers and communities” [51] contains elements of several FM milestone sub-competencies
[20]: (A) Professionalism 2: Accountability/
Conscientiousness; (B) Interpersonal communication skills
1: Patient and family-centered communication; (C)
Interpersonal communication skills 2: Interprofessional and
team communication; and (D) Systems-based practice 3:
Physician role in health systems. If a PD were condent in
their clinical competency committee’s process for assigning
milestone levels to their residents, they could decide that a
resident who achieves a level 4in these four sub-competencies meets criteria for the core outcome.
More complicated systems to meet the attestation of the
core outcomes exist. These can be considered for programs
that have been using such systems already or who have concern in the accuracy of aggregate periodic faculty assessments. In addition, such processes can be used for decisions
around entrustment for resident responsibilities. For example, Fig.11.7 shows a series of observable behaviors that can
be assessed by faculty after spending a week with a resident
on an inpatient family medicine service. An online residency
management system can compile faculty ratings into a report
like the one in Fig. 11.7. Residents might be required to
reach a level of Indirect Supervision (average score of 2.75
or greater) prior to being entrusted to take call in the hospital
without an attending present in-house. A level of Independent
might be required for the ABFM core outcome, “Diagnose
Table 11.4 Examples of assessments for the 2024 family medicine core outcomes. (From Newton etal. [51])
Core outcome Example assessments
Practice as personal physicians, providing rst contact, comprehensive
and continuity care, to include excellent doctor-patient relationships,
excellent care of chronic disease, and routine preventive care and
effective practice management
Diagnose and manage acute illness and injury for people of all ages in
the emergency room or hospital
Provide comprehensive care of children, including diagnosis and
management of the acutely ill child and routine preventive care
Develop effective communication and constructive relationships with
patients, clinical teams, and consultants
Model professionalism and be trustworthy for patients, peers, and
communities
Feedback to residents on quality of care or preventive care
Efciency of patient care assessments such as timeliness of seeing
patients, completion of charting, and coding
Preceptor and behavioral health faculty assessments of effectiveness of
doctor- patient relationship
End of clinic shift cards
Clinic eld notes
Find of inpatient hospital rotation evaluation that includes:
Efciency and thoroughness of initial assessment and oor
management
Managing discharges and other transitions of care
Effective collaboration with teammates, nurses, and other
professionals
Trustworthiness with team members and consultants
Use of multisource feedback of all members of hospital teams
Existing rotational assessments of pediatric inpatient, emergency
department, and outpatient rotations that include:
Recognition and management of emergencies
Key procedures and communication with patients, families, and
other professional on the team
Precepting assessments in continuity clinic
Likely included in all rotational assessments
Ideally develop way for CCC to monitor across rotations and
settings so can request additional assessments as necessary
Assessments from special curricula in behavioral health
Routine rotation assessments and reviews by faculty advisors or
coaches should include a component of professionalism
Recommend asking specically about trustworthiness from peers,
faculty, and rotation leads in all rotation evaluations
11 Designing Assessment toMeet theChallenge ofCompetency-Based Medical Education
99
Fig. 11.7 Example of a residency management system compiling and averaging a resident’s observable behaviors
Fig. 11.8 Assessment scale
with a sample observable
behavior
and manage acute illness and injury for people of all ages in
the emergency room or hospital” [51].
The examples thus far have demonstrated individual
pieces of an assessment system. It is important to also consider an example of an assessment system as a whole.
Although every program will need to create a system of
assessment that works for their context, this is one example
of a system that was created with a goal of implementing
CBME and has been continuously improved for more than
15years. No system is perfect. However, without examples
of functioning systems of assessment, programs are left feeling uncertain.
In the described system, assessments are completed by
faculty through an online residency management system
after experiences with residents. All assessment forms have
observable behaviors specic to the experience similar to the
example above (Fig.11.7). The assessment scale shown in
Fig.11.8 is associated with an observable behavior from a
form for precepting at the FMP.In this example, the resident
is scored as being between “with direct supervision” and
“with indirect supervision,” indicating that they required the
faculty to be present in the room with the patient to fully
meet this behavior, but they were close to needing only faculty consultation. It should be noted that there is an option
for “did not observe” for faculty to choose if they did not
have an opportunity to see this behavior in a given session.
The precepting form contains ten observable behaviors pertinent to the resident’s performance in the FMP, and each
observable behavior is followed by a free-text eld to allow
comments to add context to the assessment. The form is
completed after each FMP clinic session. Faculty
development to align expectations about the observable
behaviors and the grading scale is conducted annually.
In addition to the FMP, similar forms are completed by
faculty after experiences on an inpatient family medicine
service and a behavioral medicine clinic. Twice annually, a
global form using similar observable behaviors is disseminated to faculty, clinical staff and leadership at the FMP.All
100
observable behaviors on these forms are mapped in the
online residency management system to FM milestones.
Other forms are used after off-service rotations, but these
forms focus on rotation-specic goals. They do not feed into
the milestones in the residency management system, because
faculty members on those rotations receive different faculty
development about the assessment forms not centered on the
FM milestones. In addition, patient feedback is collected and
shared with the residents, but it does not automatically feed
into the FM milestones. All assessments are sent in the online
residency management to the resident, the advisor, and the
PD.Although time-consuming for the PD, it is essential that
they see all the assessments so that their eyes can serve as the
“lights” seeing the residents from all directions in Fig.11.3
above.
Twice per year, each resident performs a self-assessment
on all FM milestones. The residents are initially instructed
on how to use the milestones, and they are given feedback at
each self-assessment. Separately, the resident’s advisor
reviews FM milestones ratings from the observable behaviors, considering these ratings with their experience of the
resident’s competency and the qualitative comments from
faculty members, the advisor assigns a preliminary milestone score. The advisor and the resident meet [24] for a
coaching session in which they compare the resident’s selfassessment with the preliminary milestone score assigned by
the advisor in a process of guided self-assessment, an essential step in developing master adaptive learners [16]. Usually,
the advisor’s preliminary score is maintained, but at times
residents make a compelling case that the preliminary score
is incorrect. Often, this discussion reveals a halo effect in
which faculty members have overestimated a resident’s competency, but on occasion a resident can give denitive evidence that they regularly exceed the composite faculty rating.
When this occurs, an advisor may increase the preliminary
score to match the resident’s self-assessment. However, this
choice is usually coupled with a conversation about how the
resident can make sure that faculty see their competency
more clearly. This process of co-production of assessment
increases residents’ intrinsic motivation to excel [30]. Rarely,
a pattern of over- or under-estimation of resident competency is detected when faculty give feedback to the CCC.In
these cases, the mapping within the residency management
system is updated as a part of the program’s continuous
improvement efforts. The nal proposal of FM milestone ratings for the resident is plotted by the residency management
system into a “radar graph” [37] that shows the resident’s
entire milestone assessment in one image (Fig.11.9). In this
example, PGY3’s milestone scores in December of their
third year (in blue) are graphed along with the milestone ratings from the last two semesters of residency. This process
has been evaluated formally. The analysis demonstrated an
improvement in the correlation of residents’ self-assessments
D. Keister and V. Brohm
Fig. 11.9 Radar graph of a resident’s milestone assessment by reporting date
with faculty ratings and patient feedback as residents progress through their training [38].
The resident and their advisor present their proposed
milestone assignment to the CCC, which this program
calls the Resident Assessment Facilitation Team (RAFT)
[24]. The CCC makes the nal determination about the
milestone rating submitted to the GME and mediates any
disagreements between the advisor and the resident, which
are rare. The resident is also asked to reect with the RAFT
about their progress in their training and to present an individualized learning plan that they will follow through the
next semester, an essential element of the master adaptive
learning cycle [16]. A statement written by the resident
about their goals for the next semester is shared with all
residency faculty after the RAFT meeting. As residents
progress through their RAFTs, they gain condence in
their ability to plan their own educational experiences,
which will be essential at the completion of residency [16].
Figure 11.10 contrasts the plateaued minimum performance of the graduate of a training program that does not
use competency-based assessment in partnership with
coaching to develop master adaptive learners with a learner
who achieves “mastery in practice” by continuing to
develop skills beyond residency due to their educational
planning training along with emerging standards for
CBME in continuing medical education and board certication [51].
The described system of assessment takes advantage of
the ACGME’s milestones and the technology of available
residency management systems to implement CBME.The
11 Designing Assessment toMeet theChallenge ofCompetency-Based Medical Education
Fig. 11.10 The trajectory of
mastery in practice utilizing
CBME. (From [52]; Nasca
(2023). Personal
Communications)
101
CCC and the PEC of the program constantly monitor for
feedback and opportunities to improve, and members of
those committees monitor the national dialogue about the
advancement of CBME in FM in hopes of nding more
best practices to implement within the context of the
program.
The Future ofCBME
The rapid growth in the number of FM residency programs
[51], the variability in the resources available to each program [51], the rate of turnover of residency leadership [21],
and reductions in requirements for dedicated time for teaching in residencies [50] all represent challenges to the implementation of CBME. Thus, the journey to meaningfully
enact the ABFM Core Outcomes across the more than
700FM residency programs in the USA likely will last more
than a decade. However, the rst steps are in place, and the
will is present at the ABFM and ACGME to motivate this
important change [31, 51].
Holmboe shared suggestions for our discipline to accelerate the transformation of FM residencies for CBME [31].
These included creating blueprints for assessment activities
required for all programs, researching the assumptions that
have driven our competency assessments in the past, and
investing in faculty development. He also recommended specic attention given to developing innovative approaches to
assessing professionalism, practice-based learning and
improvement, and systems-based practice is important to
highlight because these ACGME core competencies have
suffered from a relative lack of attention historically.
As we previously discussed, the future of CBME in the
USA will evolve alongside similar international efforts.
Evidence around the implementation of the Can26MEDS
framework in FM suggests that GME programs in the USA
will need to increase the frequency and number of assessments of our learners to achieve the goals of CBME [52].
Additionally, the CanMEDS Framework is due to be revised
in 2025. Initial themes from the literature review for those
revisions have been published [74]. FM GME educators who
hope to anticipate the future of CBME will benet from following the progress of these revisions.
The implementation of the principles of master adaptive
learning (MAL) has been identied throughout this chapter
as a key to the success of CBME.Unfortunately, many current faculty members have not been trained as adaptive learners (Edje 2020), and therefore signicant faculty development
will be necessary. Future didactic sessions will need to
include more learner involvement and to encourage residents
to engage their metacognition (the awareness of their own
thought processes) [17, 80]. Faculty will need to be trained
as coaches to foster the “batteries” that drive MAL behaviors, and to assist residents in informed self-assessment [15,
16]. The recent ACGME requirement to require 6months of
elective time during FM residency will create exibility for
residents to guide their own learning as master adaptive
learners who will co-produce their learning experiences
(ACGME 2023, [30]).
The move to CBME also will require educators to question many of the assumptions that have formed the basis of
the GME curricular structure. For example, the organization of curricular experiences into block rotations has been
a cornerstone of medical education. It is certainly convenient logistically and is ideal under an assumption of competency after a time-based curriculum. However, concerns
from multiple approaches of thought question the block
rotational structure (Holmboe 2011). A sociological paradigm suggests a need for professional development and
training in interprofessional teams that is not supported by
block rotations. Learning theory suggests that medical
education needs to look beyond the development of com-
102
D. Keister and V. Brohm
petency in individual learners to consider how learning
occurs in “dynamic, complex and unstable clinical teams
or systems” (Holmboe 2011). Finally, examining the block
structure from a healthcare quality perspective, concerns
are raised that trainees too often rotate through dysfunctional clinical environments that do not help them to
develop the skills necessary to lead high- quality teams. If
block rotations were to be abandoned, what structure(s)
would replace them? The longitudinal integrated clinical
clerkship (LICC) model shows promise in UME [54].
Evidence supports that students trained in LICC environments maintain examination performance while being
more likely to choose a career in primary care [54, 79] and
more likely to maintain patient-centeredness without the
ethical erosion commonly seen during medical school
[57]. Longitudinal residency training has existed in FM
GME for more than 20years [44, 61], but a lack of a common denition has stied research about its effectiveness.
A consensus denition of longitudinal interleaved residency training (LIRT) was recently published [81], which
may allow further study of this option.
A time-free model of GME, in which learners graduate as
soon as they have demonstrated their competency in all facets of their role has appeal, but it would be logistically challenging [56]. Although some learners certainly can learn
content more quickly than others and even successfully
achieve competency in a discipline in less time than allotted
by the ACGME minimums [48, 56], a eld as broad as family medicine dees the suggestion that residents would have
nothing benecial to learn even if they achieve minimum
competency [50]. Therefore, a model that includes timebased systems emphasizing competency-based assessment is
more likely to emerge [55]. As has been previously mentioned in this chapter, the ACGME AIRE will have a major
role in guiding the future of FM GME [50].
Articial Intelligence (AI) is likely to transform our world
in the next decade. Certainly, it will have a role in FM GME,
as well. AI may help faculty better understand residents, customize curriculum to residents’ specic learning styles and
teach and assess more efciently [45, 68]. For example, a
recent study found that analysis of narrative assessment can
help to identify a struggling resident earlier in their training
[39]. Specically, narratives related to “1) Gaps in attention
to detail, 2) Communication decits with patients, and 3)
Difculty recognizing the ‘big picture’ in patient care” were
red ags. AI review of narrative feedback might be able to
identify themes more reliably than commonly used systems
that depend on human review. Ethical issues around the use
of AI with patient and learner data are being rapidly
addressed, and while AI can certainly carry forward the bias
of the systems that created it, there is hope that it might be
more reliable [68].
Summary
Competency-based assessment is critical to the future of FM
education, which must evolve to improve the skills of family
physicians who will assist in improving the quality of health
care in the USA.Although the evidence to support CBME is
still emerging, accreditors and certiers have decided that
the potential benet of CBME must be pursued. When built
upon a sound foundation that focuses on learner-centered
approaches to develop adaptive experts, it is hoped that a
system of assessment can more reliably create high-quality
family physicians. The ideal path to this goal is unclear. The
next decade promises to be a challenging time for FM educators. By working together toward the common goal of a better system of training, FM residency leaders will help to
shape the future of our discipline and our health system.
References
1. Accreditation Council for Graduate Medical Education. ACGME
program self-study and 10-year accreditation site visit updates.
2022, June. https://www.acgme.org/globalassets/PDFs/SelfStudy/
SS_FAQs.pdf. Accessed 13 Oct 2023.
2. Accreditation Council for Graduate Medical Education. ACGME
program requirements for graduate medical education in family
medicine. 2023a, June 1. https://www.acgme.org/globalassets/
pfassets/programrequirements/120_familymedicine_2023.pdf.
Accessed 13 Sept 2023.
3. Accreditation Council for Graduate Medical Education. Clinical
Review Environment Review (CLER) program. 2023b. https://
www.acgme.org/initiatives/clinical- learning- environment- reviewcler/. Accessed 13 Oct 2023.
4. Amiel J, Ryan MS, Andriole DA, Whelan AJ, Association of
American Medical Colleges. Core entrustable professional activities
for entering residency: summary of the 10-school pilot, 2014–2021.
2022. Available online: https://store.aamc.org/core- entrustable-
professional- activities- for- entering- residency- summary- of- the- 10school- pilot- 2014- 2021.html. Accessed 13 Sept 2023.
5. Andolsek K, Padmore J, Hauer KE, Ekpenyong A, Edgar L,
Holmboe E. ACGME clinical competency committees: a guidebook for programs. 2023. https://www.acgme.org/globalassets/
ACGMEClinicalCompetencyCommitteeGuidebook.pdf. Accessed
13 Oct 2023
6. Andreou V, Peters S, Eggermont J, Embo M, Michels NR,
Schoenmakers B. Fitness-for-purpose of the CanMEDS competencies for workplace-based assessment in general practitioner’s
training: a Delphi study. BMC Med Educ. 2023;23(1):204. https://
doi.org/10.1186/s12909- 023- 04207- 2. PMID: 37005633; PMCID:
PMC10067520.
7. Association of American Medical Colleges (AAMC). Available
online: https://www.aamc.org/about- us/mission- areas/medical-
education/cbme. Accessed 13 Sept 2023.
8. Audétat MC, Voirol C, Béland N, Fernandez N, Sanche
G. Remediation plans in family medicine residency. Can Fam
Physician. 2015;61(9):e425–34. PMCID: PMC4569126.
9. Baker SD, Turner G, Bush S.ARCH: a guidance model for providing effective feedback to learners. STFM Education Columns;
2023. https://www.stfm.org/publicationsresearch/publications/edu-
cationcolumns/2015/november/. Accessed 12 Oct 2023.
11 Designing Assessment toMeet theChallenge ofCompetency-Based Medical Education
103
10. Batalden P, Leach D, Swing S, Dreyfus H, Dreyfus S.General competencies and accreditation in graduate medical education. Health
Aff (Millwood). 2002;21(5):103–11. https://doi.org/10.1377/
hlthaff.21.5.103.
11. Bowe C, Armstrong E.Assessment for systems learning: a holistic
assessment framework to support decision making across the medical education continuum. Acad Med. 2017;92(5):585–92. https://
doi.org/10.1097/ACM.0000000000001321.
12. Campbell C, Hendry P, Delva D, Danilovich N, Kitto
S. Implementing competency-based medical education
in family medicine: a scoping review on residency programs and family practices in Canada and the United States.
Fam Med. 2020;52(4):246–54. https://doi.org/10.22454/
FamMed.2020.594402.
13. Carraccio C, Englander R, Van Melle E, ten Cate O, Lockyer J, Chan
M, Frank J, Snell L.Advancing competency-based medical education: a charter for clinician–educators. Acad Med. 2016;91(5):645–9.
https://doi.org/10.1097/ACM.0000000000001048.
14. Clements DS, Holmboe ES, Newton WP. Milestones in family
medicine: lessons for the specialty. Fam Med. 2021;53(7):618–21.
https://doi.org/10.22454/FamMed.2021.107044.
15. Cutrer WB, Atkinson HG, Friedman E, etal. Exploring the characteristics and context that allow master adaptive learners to thrive.
Med Teach. 2018;40(8):791–6. https://doi.org/10.1080/01421
59X.2018.1484560.
16. Cutrer, etal. The master adaptive learner: from the AMA MedEd
innovation series. 1st ed. Elsevier, Philadelphia; 2020.
17. Edje L, Price DW. Training future family physicians to become
master adaptive learners. Fam Med. 2021;53(7):559–66. https://
doi.org/10.22454/FamMed.2021.192268.
18. Eno C, Correa R, Stewart N, Lim J, Westerman ME, Holmboe ES,
Edgar L.Milestones guidebook for residents and fellows. ACGME;
2020. https://www.acgme.org/globalassets/PDFs/Milestones/
MilestonesGuidebookforResidentsFellows.pdf. Accessed 10 Aug
2023.
19. Epstein RM, Siegel DJ, Silberman J. Self-monitoring in clinical
practice: a challenge for medical educators. J Contin Educ Heal
Prof. 2008;28(1):5–13. https://doi.org/10.1002/chp.149.
20. Family Medicine Milestones Workgroup. Family medicine milestones – ACGME. 2019. https://www.acgme.org/globalassets/
PDFs/Milestones/FamilyMedicineMilestones.pdf. Accessed 13
Sept 2023.
21. Fernald D, Hester CM, Brown SR. Why family medicine program directors leave their position. Fam Med. 2021;53(5):347–54.
https://doi.org/10.22454/FamMed.2021.746153.
22. Fine LG. The SWOT analysis: using your strength to overcome
weaknesses, using opportunities to overcome threats. Kick It; 2010.
23. Flynn T, etal. The core entrustable professional activities (EPAs)
for entering residency. AAMC; 2014. https://www.aamc.org/
about- us/mission- areas/medical- education/cbme/core- epas
24. Foster E, Biery N, Dostal J, Larson D. RAFT (resident assessment facilitation team): supporting resident well-being through
an integrated advising and assessment process. Fam Med.
2012;44(10):731–4.
25. Foundational competencies for Undergraduate Medical
Education. UME Competencies AAMC | AACOM | ACGME.
2023. https://cloud.email.aamc.org/UME- Competencies-
AAMC- AACOM- ACGME
26. Frank JR, Snell LS, Cate OT, etal. Competency-based medical education: theory to practice. Med Teach. 2010;32(8):638–45. https://
doi.org/10.3109/0142159X.2010.501190.
27. Frank JR, Snell L, Sherbino J, editors. CanMEDs 2015 physician
competency framework; 2015. p. 1–30. Available from https://
www.royalcollege.ca/ca/en/canmeds/canmeds- framework.html.
Accessed 15 Oct 2023.
28. Frazier W, Wilson SA, D’Amico F, Bergus GR.Resident remediation in family medicine residency programs: a CERA survey
of program directors. Fam Med. 2021;53(9):773–8. https://doi.
org/10.22454/FamMed.2021.546572.
29. Gunja MZ, Gumas ED, Williams RD II. U.S. health care from
a global perspective, 2022: accelerating spending, worsening outcomes. Commonwealth Fund, Jan 2023. https://doi.
org/10.26099/8ejy- yc74
30. Holmboe ES. Work-based assessment and co-production in postgraduate medical training. GMS.J Med Educ. 2017;34(5):Doc58.
https://doi.org/10.3205/zma001135. PMID: 29226226; PMCID:
PMC5704603
31. Holmboe ES.The transformational path ahead: competency-based
medical education in family medicine. Fam Med. 2021;53(7):583–9.
https://doi.org/10.22454/FamMed.2021.296914.
32. Holmboe ES, Iobst W.Accreditation Council for Graduate Medical
Education: ACGME assessment guidebook. 2020. https://www.
acgme.org/globalassets/pdfs/milestones/guidebooks/assessmentguidebook.pdf. Accessed 9 Oct 2023.
33. Holmboe ES, Yamazaki K, Nasca TJ, Hamstra SJ. Using longitudinal milestones data and learning analytics to facilitate the
professional development of residents: early lessons from three
specialties. Acad Med. 2020;95(1):97–103. https://doi.org/10.1097/
ACM.0000000000002899.
34. Humphrey-Murto S, Wood TJ, Ross S, Tavares W, Kvern B, Sidhu
R, Sargeant J, Touchie C.Assessment pearls for competency-based
medical education. J Grad Med Educ. 2017;9(6):688–91. https://
doi.org/10.4300/JGME- D- 17- 00365.1. PMID: 29270255; PMCID:
PMC5734320.
35. Institute of Medicine, Committee on Quality of Health Care in
America. Crossing the quality chasms: a new health system for the
21st century. Washington: National Academy Press; 2001.
36. Iobst WF, Holmboe ES. Programmatic assessment: the secret
sauce of effective CBME implementation. J Grad Med Educ.
2020;12(4):518–21. https://doi.org/10.4300/JGME- D- 20- 00702.1.
PMID: 32879699; PMCID: PMC7450746.
37. Keister DM, Larson D, Dostal J, Baglia J. The radar graph: the
development of an educational tool to demonstrate resident competency. J Grad Med Educ. 2012;4(2):220–6. https://doi.org/10.4300/
JGME- D- 11- 00163.1.
38. Keister DM, Hansen SE, Dostal J.Teaching resident self- assessment
through triangulation of faculty and patient feedback. Teach Learn
Med. 2017;29(1):25–30. https://doi.org/10.1080/10401334.2016.1
246249.
39. Kelleher M, Kinnear B, Sall DR, Weber DE, DeCoursey B, Nelson
J, Klein M, Warm EJ, Schumacher DJ. Warnings in early narrative assessment that might predict performance in residency: signal
from an internal medicine residency program. Perspect Med Educ.
2021;10(6):334–40. https://doi.org/10.1007/s40037- 021- 00681- w.
Epub 2021 Sep 2. PMID: 34476730; PMCID: PMC8633188.
40. Kohn LT, Corrigan J, Donaldson MS. To err is human: building
a safer health system. Washington, DC: National Academy Press;
2000.
41. Koster AS, Mantel-Teeuwisse AK, Woerdenbag HJ, etal. Alignment
of CanMEDS-based undergraduate and postgraduate pharmacy
curricula in The Netherlands. Pharmacy. 2020;8(3):117. https://doi.
org/10.3390/pharmacy8030117.
42. Lovell E, Mejicano G, etal. The coalition for physician accountability’s undergraduate medical … recommendations for comprehensive improvement of the UME-GME transition. 2021, August 26.
https://physicianaccountability.org/wp- content/uploads/2021/08/
UGRC- Coalition- Report- FINAL.pdf. Accessed 13 Sept 2023.
43. McGaghie WC, Miller GE, Sajid AW, Telder TV. Competencybased curriculum development in medical education. Public health
paper no. 68. Geneva: World Health Organization; 1978.
104
D. Keister and V. Brohm
44. Merenstein D, Damico F, Devine B, et al. Longitudinal versus
traditional residencies: a study of continuity of care. Fam Med.
2001;33(10):746–50.
45. Mir MM, Mir GM, Raina NT, Mir SM, Mir SM, Miskeen E,
Alharthi MH, Alamri MMS. Application of articial intelligence in medical education: current scenario and future perspectives. J Adv Med Educ Prof. 2023;11(3):133–40. https://doi.
org/10.30476/JAMP.2023.98655.1803. PMID: 37469385; PMCID:
PMC10352669.
46. Mitchell D. Family medicine welcomes largest class
ever. AAFP. 2023, March 17. https://www.aafp.org/news/
education- professional- development/2023- match- day.
html#:~:text=Family%20medicine%20programs%20filled%20
4%2C530,record%2Dbreaking%20class%20in%202021
47. Morris JL, Freiger BG, Pickens TS, Willett LL. Thriving during
the 10-year accreditation site visit: practical tips from a program’s
perspective. J Grad Med Educ. 2018;10(6):626–8. https://doi.
org/10.4300/JGME- D- 18- 00121.1.
48. Murray KE, Lane JL, Carraccio C, et al. Education in Pediatrics
Across the Continuum (EPAC) study group. Crossing the gap:
using competency-based assessment to determine whether
learners are ready for the undergraduate-to-graduate transition. Acad Med. 2019;94(3):338–45. https://doi.org/10.1097/
ACM.0000000000002535.
49. Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME
accreditation system – rationale and benets. N Engl J Med.
2012;366(11):1051–6. https://doi.org/10.1056/NEJMsr1200117.
Newton WP, Hoekzema G, Magill M, Fetter J, Hughes L. The
promise of Aire. Ann Fam Med. 2022a;20(4):389–91. https://doi.
org/10.1370/afm.2869.
50. Newton WP, Hoekzema G, Magill M, Hughes L.Dedicated time for
education is essential to the residency learning environment. J Am
Board Fam Med. 2022;35:1035–7.
51. Newton WP, Magill M, Barr W, Hoekzema G, Karuppiah S,
Stutzman K. Implementing competency based ABFM board eligibility. J Am Board Fam Med. 2023a;36(4):703–7. https://doi.
org/10.3122/jabfm.2023.230201R0.
52. Newton W, etal. AFMRD webinar: core outcomes and competency
based ABFM board eligibility. 2023b, August 14. https://vimeo.
com/854427126
53. Norcini J, Anderson MB, Bollela V, etal. Consensus framework for
good assessment. Med Teach. 2018;2018:1–8.
54. Norris TE, Schaad DC, DeWitt D, Ogur B, Hunt DD.Consortium
of longitudinal integrated clerkships. Longitudinal integrated clerkships for medical students: an innovation adopted by medical
schools in Australia, Canada, South Africa, and the United States.
Acad Med. 2009;84:902–7.
55. Nousiainen MT, Caverzagie KJ, Ferguson PC, Frank JR, ICBME
Collaborators. Implementing competency-based medical education: what changes in curricular structure and processes are needed?
Med Teach. 2017;39(6):594–8. https://doi.org/10.1080/01421
59X.2017.1315077.
56. Nousiainen MT, Mironova P, Hynes M, etal. Eight-year outcomes
of a competency-based residency training program in orthopedic
surgery. Med Teach. 2018;40(10):1042–54. https://doi.org/10.1080
/0142159X.2017.1421751.
57. Ogur B, Hirsh D, Krupat E, Bor D. The Harvard Medical
School-Cambridge integrated clerkship: an innovative model of
clinical education. Acad Med. 2007;82(4):397–404. https://doi.
org/10.1097/ACM.0b013e31803338f0.
58. Philibert I, Lieh-Lai M. A practical guide to the ACGME selfstudy. J Grad Med Educ. 2014;6(3):612–4. https://doi.org/10.4300/
JGME- 06- 03- 55. PMID: 26279804; PMCID: PMC4535243.
59. Phillips RL Jr, George BC, Holmboe ES, Bazemore AW, Westfall
JM, Bitton A.Measuring graduate medical education outcomes to
honor the social contract. Acad Med. 2022;97(5):643–8. https://doi.
org/10.1097/ACM.0000000000004592. Epub 2022 Apr 27. PMID:
35020616; PMCID: PMC9028305
60. Plan-Do-Study-Act (PDSA) Directions and Examples. Content
last reviewed September 2020. Agency for Healthcare Research
and Quality, Rockville, MD. https://www.ahrq.gov/health- literacy/
improve/precautions/tool2b.html
61. Reust CE.Longitudinal residency training: a survey of family practice residency programs. Fam Med. 2001;33(10):740–5.
62. Robbins JB, Sarkany D.Self-study: practical tips for a successful
and rewarding experience. Acad Radiol. 2017;24(6):721–4.
63. Sargeant J, Armson H, Chesluk B, Dornan T, Eva K, Holmboe E,
Lockyer J, Loney E, Mann K, van der Vleuten C. The processes
and dimensions of informed self-assessment: a conceptual model.
Acad Med. 2010 Jul;85(7):1212–20. https://doi.org/10.1097/
ACM.0b013e3181d85a4e.
64. Saultz J.Competency-based education in family medicine residency
education. Fam Med. 2021;53(7):590–2. https://doi.org/10.22454/
FamMed.2021.816448.
65. Shadid AM, Bin Abdulrahman AK, Bin Dahmash A, et al.
SaudiMEDs and CanMEDs frameworks: similarities and differences. Adv Med Educ Pract. 2019;10:273–8. https://doi.
org/10.2147/AMEP.S191705.
66. Shaughnessy AF, Sparks J, Cohen-Osher M, Goodell KH, Sawin
GL, Gravel J.Entrustable professional activities in family medicine. J Grad Med Educ. 2013;5(1):112–8.
67. Shaw E, Oandasan I, Fowler N, editors. CanMEDS-FM 2017: a
competency framework for family physicians across the continuum. Mississauga: The College of Family Physicians of Canada;
2017.
68. Sun L, Yin C, Xu Q, Zhao W. Articial intelligence for
healthcare and medical education: a systematic review. Am J
Transl Res. 2023;15(7):4820–8. PMID: 37560249; PMCID:
PMC10408516
69. ten Cate O. Competency-based education, entrustable professional activities, and the power of language. J Grad Med Educ.
2013;5(1):6–7.
70. ten Cate O, Scheele F.Competency-based postgraduate training:
can we bridge the gap between theory and clinical practice? Acad
Med. 2007;82(6):542–7.
71. Ten Cate O, Chen HC, Hoff RG, Peters H, Bok H, van der Schaaf
M. Curriculum development for the workplace using Entrustable
Professional Activities (EPAs): AMEE guide no. 99. Med Teach.
2015;37(11):983–1002.
72. The Association of Family Medicine Residency Directors.
Entrustable professional activities (EPAS). AFMRD EPAs. 2015.
https://www.afmrd.org/page/epa
73. The family medicine milestone project. J Grad Med Educ. 2014;6(1
Suppl 1):74–86. https://doi.org/10.4300/JGME- 06- 01s1- 05. PMID:
24701268; PMCID: PMC3966595.
74. Thoma B, Karwowska A, Samson L, Labine N, Waters H, Giuliani
M, Chan TM, Atkinson A, Constantin E, Hall AK, GomezGaribello C, Fowler N, Tourian L, Frank J, Anderson R, Snell
L, Van Melle E.Emerging concepts in the CanMEDS physician
competency framework. Can Med Educ J. 2023;14(1):4–12.
https://doi.org/10.36834/cmej.75591. PMID: 36998506; PMCID:
PMC10042782.
75. Van Aerde J, Gomes MM, Giuliani M, Thoma B, Lieff S.Complex
adaptive systems in CanMEDS 2025. Can Med Educ J.
2023;14(1):50–3. https://doi.org/10.36834/cmej.75538. PMID:
36998500; PMCID: PMC10042785.
76. Van Der Vleuten CP. Revisiting ‘Assessing professional competence: from methods to programmes’. Med Educ. 2016;50(9):885–8.
https://doi.org/10.1111/medu.12632.
11 Designing Assessment toMeet theChallenge ofCompetency-Based Medical Education
105
77. Van Der Vleuten CP, Schuwirth LW, Driessen EW, Dijkstra J,
Tigelaar D, Baartman LK, van Tartwijk J.A model for programmatic assessment t for purpose. Med Teach. 2012;34(3):205–14.
https://doi.org/10.3109/0142159X.2012.652239.
78. Van Melle E, Frank JR, Holmboe ES, Dagnone D, Stockley
D, Sherbino J. A core components framework for evaluating
implementation of competency-based medical education programs. Acad Med. 2019;94(7):1002–9. https://doi.org/10.1097/
ACM.0000000000002743.
79. Woloschuk W, Myhre D, Jackson W, McLaughlin K, Wright
B.Comparing the performance in family medicine residencies of
graduates from longitudinal integrated clerkships and rotationbased clerkships. Acad Med. 2014;89(2):296–300. https://doi.
org/10.1097/ACM.0000000000000113.
80. Zakrajsek T, Newton WP.Promoting active learning in residency
didactic sessions. Fam Med. 2021;53(7):608–10. https://doi.
org/10.22454/FamMed.2021.894932.
81. Zeller TA, Beben K, Kong M, Martonffy I, Patterson S, Deas W,
Heo M, Keister DM.Longitudinal interleaved residency training:
a consensus denition. Fam Med. 2023;55(5):311–6. https://doi.
org/10.22454/FamMed.2023.378423.
Core Competencies, Milestones, and Entrustable Professional Activities
KrisEmilyMcCrory
12
Key Points
• Core Competencies and Milestones should form the foundation of Family Medicine GME training.
• Work-based assessments such as Entrustable Professional
Activities can bridge the ideals of the Core Competencies
with real work practice.
• The ACGME and ABFM continue to shift emphasis
to outcome-based medical education with Core
Competencies and Milestones.
• Entrustment decisions shift the assessment focus from
past performance to prospective judgment of readiness
for future autonomy.
• Core Competencies and Milestones can be used as basis
for curriculum assessment.
• Clinical Competency Committees have responsibilities
beyond individual resident assessment and the Core
Competencies and Milestones are the foundation of their
charge.
• Core Competencies and Milestones provide guideposts to
allow programs exibility to create individualized learning plans and program components.
• Implementing learner centered assessment processes
within the frame work of the Core Competencies and
Milestones can be used to develop skills of master adaptive learners.
Educational Rationale ofCore Competencies
andMilestones
Previous to the implementation of Core Competencies and
Milestones, GME focused on two elements: the content studied and the examinations designed to assess student knowl-
K. McCrory (*)
Cheshire Medical Center-Dartmouth Health Family Medicine
Residency Program, Keene, NH, USA
edge of that content [13]. This focus emphasized trying to
identify what learners should be taught in order to correctly
answer test questions as a surrogate for competency [12].
Curriculum in this model frequently involves rotating
through a myriad of clinical rotations, with an emphasis on
standardized learning experiences and time spent in training
punctuated with a limited number of high stakes, more often
summative than formative, assessments [22]. In this model,
the learner’s attention focuses on serving their time and
passing the necessary exams with little or no time spent in
self- reection or receiving regular meaningful, formative
feedback. This emphasis leaves little to no focus on the
actual outcome, in this case, a physician capable of providing safe and effective care within the complex healthcare
system. Rather, the goal is passing the test [13].
Calls for a curricular and assessment paradigm shift in
medical training came as early as the 1970s with the publication of “Competency-based Curriculum Development in
Medical Education” [25]. The authors identied a need for
intentionally designed medical training focused on meeting
the public health needs of society. At that time there was little movement in medical education to undertake such a monumental shift. Pressure would mount, however, in the
following decades with the undeniable realization that medical quality and safety issues plagued the health care system.
Data from the Institute of Medicine’s To Err is Human [21]
and Crossing the Quality Chasm [19] spurred an urgent call
from the public for substantial reform of medical training.
The previous training model of completing an arbitrary number of clinical rotations capped by high-stakes examination
that depended heavily on an archaic notion that medical residents learned at the same pace and in roughly the same way
to end up at the same place when training concludes was
clearly not working [16].
The Accreditation Council of Graduate Medical Education
(ACGME) in conjunction with the American Board of
Medical Specialties (ABMS) launched the Outcome Project
in 2001 [32]. While acknowledging that time spent in clini-
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2025
R. Kellerman, G. Irwin (eds.), Graduate Medical Education in Family Medicine, Excellence in Medical Education 2,
https://doi.org/10.1007/978-3-031-70741-4_12
107
108
K. McCrory
cal experiences and exposure matter, it is insufcient to
ensure competent family physicians capable of providing
safe, quality healthcare to the public [26]. The ACGME and
ABMS demanded greater accountability of programs to
society with a more explicit link between graduate medical
education and clinical outcomes. A central goal of medical
education must be producing physicians prepared to practice
in their communities and all training must consider the specic needs of society [17, 31].
Together the ACGME and ABMS identied six domains
of core competencies (Box 12.1) as anchor points for the curriculum and assessment [32]. The end goal of the Outcomes
Project was to “foster resident physicians’ development in
competencies in these six domains and collect performance
data that reliably and accurately depicts residents’ ability to
care for patients and to work effectively in healthcare delivery systems” with the assumption “that quality patient care
results when residents acquire and apply competencies effectively [32].
Box 12.1 ACGME Core Competency Domains
Patient Care
Medical Knowledge
Systems-Based Practice
Practice-Based Learning and Improvement
Professionalism
Interpersonal and Communication Skills
ACGME [1]
Beginning in 2013, the ACGME rolled out the implementation of specialty-specic milestones requirements for all
accredited graduate medical training programs. Over the
next decade residency and fellowship programs incorporated
educational solutions and strategies to ensure residents demonstrated mastery of these domains of competency. The
ACGME moved forward with Family Medicine Milestones
2.0in 2019, addressing initial limitations, reducing unnecessary complexity, and improving community engagement [8].
The implementation of the Core Competencies and
Milestones initiated the rst steps in a shift by the ACGME
and ABFM toward an outcome or competency-based educational paradigm and an indication that educational outcomes
would be key going forward. The ACGME advised the
Residency Review Committees (RRC) would “take into
account residency programs’ educational outcomes when
making accreditation decisions instead of relying solely on
programs’ descriptions of their resources and educational
policies and processes” [32]. It is insufcient to have a great
curriculum on paper. Indeed “consideration of the outcomes
should be the basis for curriculum development and evaluation” [13]. If the curriculum fails to produce competent physicians, it is rendered meaningless. The Core Competencies
and Milestones provide assessment anchors and curriculum
guideposts to direct programs into the realm of competencybased medical education as championed by the ACGME and
the ABFM.
Dening Core Competencies andMilestones
The Core Competencies integrate the vast knowledge, skills,
and attitudes to be mastered during training through knowledge acquisition and practice experience with intentional
formative feedback. [14, 39]. A framework of such professional qualities should be specic and comprehensive enough
that they can be intentionally taught and measured [6, 34].
Each domain of competency is divided into Milestones, or a
continuum of levels residents progress through as they
develop mastery of the Core Competencies. Each Core
Competencies stand equal to the others. One cannot compensate a lack in one area with excellence in another. Physicians
must possess an adequate body of knowledge, be able to
apply that knowledge in service to others, conduct themselves professionally, work effectively in teams, communicate compassionately with patients and respectfully with
colleagues, collaborate to improve systems of care, and
engage in critical reection and lifelong learning [23].
Together these domains of competency form a high-level
blueprint for the curriculum [41].
While the Core Competencies describe the domains in
which physicians must demonstrate satisfactory knowledge,
skills, and attitudes, Milestones are the developmental stages
within each of those domains. Milestones are markers along
a stepwise progression of expertise that can be both observed
and assessed [10]. Whereas Core Competencies are consistent across all specialties, the Milestones, particularly in the
domains of medical knowledge and patient care are unique
to each specialty. All physicians must demonstrate competence in medical knowledge, but the specic components of
that may look different for family medicine compared to
general surgery or obstetrics-gynecology, for example.
Family Medicine Milestones dene and rene the core
abilities of the family medicine specialty, allowing the implementation of outcomes within a competency framework
[16]. As an example, within Family Medicine Milestones
2.0, published in 2019, Medical Knowledge is broken down
into two sub-competency domains: “Demonstrates medical
knowledge of sufcient breadth and depth to practice family
medicine” and “Critical thinking and decision making.”
Within each of these sub-competency domains, there are one
to four specic milestones provided to guide clinical educa-