Добавил:
Sekretar
kiopkiopkiop18@yandex.ru
t.me/Prokururor I Вовсе не секретарь, но почту проверяю
Опубликованный материал нарушает ваши авторские права? Сообщите нам.
Вуз:
Предмет:
Файл:Ординатура / Хирургия / Библиотека им академика М.И. Перельмана / Книга_2823_Библиотеки_им_академика_М_И_Перельмана
.pdf
considered when integrating SPs into an assessment program
and include direct and indirect costs, the need for robust SP
and faculty development, and the limits of simulation. The
additional need to assess for validity evidence is discussed
separately (see “Applying Kane’s Validity Framework”).
Expense of Standardized Patient
Programs
Concerns regarding the expense of SP-based assessments
understandably and appropriately curtail their use.
19,22,41
Research suggests that it may be difficult to capture the full
range of direct and indirect costs associated with OSCEs, and
that higher-stakes exams and initial administrations are more
expensive than subsequent ones.41 Costs may include faculty,
staff, and center time as well as access to testing
infrastructure, software platforms, and simulation
equipment. Hiring SP trainers and SPs can be expensive and,
depending upon the location of the program, recruitment
may be challenging.
Logistic Complexity and Indirect Costs
Likewise, finding the time and space to conduct SP-based
teaching and assessment activities may be difficult for
programs that do not have clinical skills centers or are not
affiliated with institutions where such centers are located.
Program directors must also consider the logistic complexity
of the assessment. Multistation OSCES are likely to require
several days or weeks to administer, depending on the
number of participants and stations, necessitating that small
cohorts of students be made available on a rotating basis and
https://t.me/med1917
impacting other curricular experiences during the test period.
One potential remedy to resource limitations is collaboration
among programs and institutions to combine resources and
expertise, minimizing financial and other resource
commitments and ensuring the availability of SPs, case
material, and trained personnel. The development of regional
consortia focusing on the use of SPs (and other simulation
methods) in education and assessment is one such approach
that is well described.
12,42
Rater Training and Bias
The balance of faculty and SPs for observation and feedback
highlights the need to consider how raters are assigned and
trained within the assessment program to best advantage.
Implementation of SP-based teaching and assessment in
educational programs may decrease the need for faculty to
participate in the more routine aspects of assessment and
may better prepare SPs for their teaching and assessment
roles.
19,40
Well-trained SPs can provide reliable assessment
and feedback on interviewing and physical examination
skills at the time of the assessment, allowing faculty to focus
on developing and maintaining the case bank, standard
setting, and quality assurance and on grading diagnostic or
therapeutic management, case presentations, and problem-
solving.40 Faculty development efforts should ensure that
faculty are familiar with expectations for observation and
rating skills and not assume that faculty familiarity with
workplace-based assessments (WBAs) will transfer to the
simulation center or clinical skills laboratory. Faculty require
training to ensure that ratings are reliable, and program
quality assurance should include assessments of interrater
https://t.me/med1917
reliability and bias, which has the potential to impact scoring
based on gender, race/ethnicity, and primary spoken
language other than English.
43
,44
Rater training is discussed
in more detail later in this chapter.
Limitations of the Simulation
Environment
While the simulated encounter approximates a true clinical
encounter, it is important to acknowledge its limitations,
including the challenges in simulating certain findings, the
effect of observation on the demonstration of skills, and
uncertainty regarding how the learner would interpret
pathologic findings in a real patient encounter. While task
trainers offer opportunities to include an increasing array of
findings (such as acute or chronic papilledema represented
alongside an SP portrayal of a patient presenting with
headache), the inclusion of the simulated finding prompts the
student to anticipate the abnormal finding and does not
necessarily measure the ability to recognize pathologic
findings when present in real patients.
45
In addition, other
physical examination findings (such as an anterior drawer
sign in a patient with a knee injury) cannot be simulated and
may undermine the ability to assess certain competencies
using SPs. In this respect, SPs should be considered a
supplement to rather than a replacement for assessing
competence with real patients. Indeed, to ensure that a broad
spectrum of physical findings are assessed within an
educational program, a combination of real and simulated
findings would be ideal,19 keeping in mind that a high
volume of “real” patient contact is still required for a trainee
to appreciate the diversity of clinical presentations. The
https://t.me/med1917
multilayered nature and social implications of real patient
encounters may contribute to a learning effect not captured
during encounters with simulated patients.
46
Following on this theme, it is important to remember that
the performance of students and residents in assessment
environments may not define their capacity to perform in
real clinical settings. In fact, data describing the use of SPs in
assessing practicing physician competence suggest that there
may be a significant difference between demonstrated
capacity to perform (competence) and actual performance in
practice, particularly when challenges with efficiency are not
taken into account.
47,48
There is clearly an observation
(“Hawthorne”) effect operating under such simulated
conditions, and a station that invites the student to
demonstrate focused, discrete tasks risks encouraging
learners to compartmentalize skills to perform well on the
assessment rather than well in the clinical setting. For
example, the ability to interpret an ECG could be better
assessed using a written assessment, but the ability to elicit a
relevant history for chest pain, perform a cardiac
examination, and explain the need for an ECG to a patient
better approximates clinical experiences. Conversely, asking
too much of a learner in a time-limited OSCE station risks
creating a cognitive load that is unrealistically challenging
and may undermine the validity of the assessment.
Ultimately, the demonstration of competence in simulation
should not be taken as final, absolute evidence that such
abilities will be exercised in real clinical settings. Data on the
correlation between OSCE scores and other clinical skills
performance measures are variable and ultimately
challenging to collect.
49,50
Still, assessment programs should
https://t.me/med1917
monitor alignment with future exam and real-world
performance and incorporate WBAs that ensure that relevant
knowledge, skills, and behaviors are manifest in the day-today environment of the future physician, as well (see
discussion of validity later in this chapter).
34,35
The implications of the strengths and weaknesses for SPbased assessments is that, ultimately, SP assessments should
be reserved to assess only those skills that cannot be better
assessed in other ways.22 Creating a program of SP-based
assessments with appropriate scope, breadth, and depth
requires an understanding of the purpose of each assessment
and how it fits into the broader assessment program.
Planning for Standardized PatientBased Assessments
When planning an SP assessment it is essential to know how
it functions within the broader program of assessment (see
Fig. 6.1). Principles guiding the development of an
assessment program are well described elsewhere (see
Chapter 3), but should begin with a centrally coordinated
vision guided by a competency framework in which each
assessment functions as “a tool fit for a purpose.”
27,51–57
A
well-designed program will include a range of instruments, a
range of stakes to enhance learning while enabling decisions
about progression, and the collection of data to inform
program evaluation.58 For SP-based assessments,
considerations include (1) how SP-based assessments are
mapped to a competency framework, (2) which competencies
to assess using SPs, and (3) the impact of assessment for and
of learning on the design.
https://t.me/med1917
Mapping to a Competency Framework
Mapping SP-based assessments within the program of
assessment benefits from the adoption of a competency
framework, of which there are several to consider (see
Chapter 1, Table 1.3), including the Accreditation Council of
Graduate Medical Education/American Board of Medical
Specialties (ACGME/ABMS) core competencies,
59,60
Canadian Medical Education Directives for Specialists
(CanMEDS) physician competency framework,61 and United
Kingdom General Medical Council’s Outcomes for
Graduates.62 Institutions may also choose to create their own
milestone or competency framework, or supplement an
existing competency framework—for example, by adding
relevant subcompetencies (e.g., components within patient
care that are specific to the clinical skills of history and
physical examination or that highlight which competencies
relate to knowledge and comprehension versus application,
analysis, synthesis, and evaluation).
28,63,64
With the competency framework chosen, programs can
embed longitudinal milestones divided into progressively
advancing skill levels (proficiency levels) as described by the
Dreyfus model of skill acquisition: novice, advanced
beginner, competent, proficient, expert65 (see Chapter 1,
Table 1.4). Educators are then challenged to develop SP-
based assessments to inform learner progress in milestones
achievement, which may have implications for item
development as well as the complexity of the case design.
64
For example, in terms of item development, an early learner
being assessed for medical history (within the competency of
patient care) may function as an “advanced beginner” by
inquiring about the core characteristics of the chief concern
https://t.me/med1917
(e.g., asked about the time course, what makes it better, what
makes it worse, etc.) and may demonstrate a higher level of
“competence” by also asking about pertinent positives and
negatives to guide diagnostic reasoning. To assess for a
higher degree of competence in the medical history, one
could include a more challenging syndrome in a subsequent
SP-based assessment that requires consideration for special
patient populations, or introduce a communication challenge.
Mapping each subcompetency to the longitudinal milestones
framework helps ensure that each assessment is appropriate
for the learner level.
28
,66
SP-based assessment should not be
relied upon as the sole assessment method to inform a
judgment on a particular milestone. When used as one
component in a multifaceted assessment program, however,
SP scenarios may be useful to determine where learners are
at in terms of knowledge, skills, and attitudes prior to
entering a new phase of training such as internship.
63
Which Competencies to Assess With
Standardized Patients
With a competency framework in place, assessment tools can
be combined to create a well-rounded and robust program
taking factors such as cost-effectiveness, acceptability,
reliability, validity, and educational impact into account
when choosing which tools to deploy.56 Using the ACGME
competencies as an example, assessment of Interpersonal and
Communication Skills and the patient-centered elements
(eliciting patient information, gathering essential and
accurate information from patients, and counseling and
educating patients) within the Patient Care domain will
necessarily rely more heavily on assessments that include
https://t.me/med1917
observation of the clinical encounter, both in simulation and
in clinical care. Standardized patients may also be used to
assess behaviors intrinsic to the Professionalism domain,
including responsiveness to patients’ culture, age, gender,
health literacy, spiritual beliefs, and intellectual and physical
disabilities.
67–70
The flexibility of SP-based methods also facilitates
development of assessment exercises to measure competence
in Practice-based Learning and Improvement and Systemsbased Practice, which are notoriously challenging to measure.
Faculty, clerkship, and program directors can work with SP
training staff to develop cases to assess trainees’ abilities to
search, appraise, and apply scientific evidence in patient
management; use information technology in managing
patient information and online resources; elicit patient
preferences in making informed decisions; and provide
appropriate care for selected conditions and health
maintenance (see Chapter 10).2 Over the past decade, health
professions educators have focused their attention on the
development of SP-based methods that measure
competencies essential for achieving current healthcare
priorities, particularly those related to patient safety,
transitions of care, interprofessionalism, and working in
teams.
71,72
SP-based assessment scenarios may also be
created to assess the sociocultural dimensions of the patient
safety competency including “near-miss” management and
medical error disclosure.
73–75
Standardized patients in the role of a standardized
healthcare professional may be trained to receive a patient
handoff,
76–78
or participate in an interdisciplinary team
meeting designed to measure collaborative skills when
https://t.me/med1917
developing a patient care plan.
79–81
Simulated
interprofessional teams can also be created to assess
performance of team-oriented behaviors including ability to
manage conflict, advocate for the patient, and speak up
against a power gradient.82 A tool with high-quality validity
evidence is available for assessment of these collaborative
practice competency domains.83 Using SP-based
methodology to assess competency in these more complex
health systems science constructs can be a challenge, as case
scenarios by nature of their complexity may be more difficult
to standardize, require the development of additional rating
scales, and require additional resources including more than
one SP involved in the station.
Assessment tools for the Medical Knowledge domain are
likely to include paper- or online-based assessments that
allow learners to describe their thinking outside the clinical
encounter (see Chapter 7 for a discussion of other methods).
While SP-based methods are not particularly efficient when
applied to the evaluation of Medical Knowledge, they may be
used to measure the application of knowledge in simulated
patient care situations (see Chapter 7). The SP encounter can
be supplemented with structured global ratings such as the
mini clinical evaluation exercise (mini-CEX),84 or, to tap other
important skills, combined with a patient note or some form
of chart stimulated recall process. Since these types of
evaluation approaches model those currently used in real
clinical settings, they are more readily accepted, both by
those who are responsible for the assessment and those who
are being assessed. Here, program or clerkship directors may
also decide to employ SPs for assessment of knowledge
application in important medical cases that are infrequently
https://t.me/med1917
or inconsistently encountered during clinical rotations.
Implications of High- and Low-Stakes
Assessments
SP-based assessment methods may be used for the purpose
of summative assessment, in which the goal is to determine
the proficiency of a learner against established standards, and
their readiness to graduate or to move on to the next level of
training; or formative assessment, in which the emphasis is
on monitoring student learning and providing feedback (see
Fig. 6.1). Summative assessment, which usually occurs at the
conclusion of a phase of training, has been termed assessment
of> learning, whereas formative assessment typically is
embedded within the instructional cycle and referred to as
assessment for> learning.52 Programmatic assessment (see
Chapter 3) describes an approach to assessment that takes
advantage of multiple assessment tools and data points to
optimize both learning and the development of competence.
Assessments should occur longitudinally with assessment for
learning opportunities placed at higher frequency within the
instructional cycle and higher-stakes summative assessments
based on a larger number of assessment data points.
51
SP assessments generate a variety of assessment data, all of
which may be used for assessment purposes but may be
more or less appropriate depending on the stakes of the
assessment. For example, within SP methodology, feedback
can include face-to-face or written narrative feedback from
faculty and/or SPs, or as part of a group debriefing session.
Case-specific content checklists can be used to determine
whether the trainee met expected performance standards for
data gathering. The same checklist can serve as a discussion
https://t.me/med1917
Соседние файлы в папке Библиотека им академика М.И. Перельмана