Добавил:
Sekretar
kiopkiopkiop18@yandex.ru
t.me/Prokururor I Вовсе не секретарь, но почту проверяю
Опубликованный материал нарушает ваши авторские права? Сообщите нам.
Вуз:
Предмет:
Файл:Ординатура / Хирургия / Библиотека им академика М.И. Перельмана / Книга_112_библиотеки_им_акад_М_И_Перельмана
.pdf
contains a 4-level scale: level 1 = show and tell; level 2 = active
help (“smart help”); level 3 = passive help (“dumb help”);
and level 4 = supervision only (“no help”).
33
Fig. 4.5 The Zwisch Proposed Model for Teaching and
Assessment in the Operating Room.
Achieving Construct Alignment Through
Simplicity
One way to enhance the construct alignment of newer, more
granular assessment methods is to locate them within the
rhythm of observation-reflection-action also seen in the
Subjective-Objective-Assessment-Plan (SOAP) format and in
RIME. In this way the RIME framework can function as an
organizer that allows assimilation or subsumption of more
elaborate structures under something already familiar. One
https://t.me/med1917
example of this locates the 13 EPAs recently proposed by the
AAMC as a set of tasks in which all students should be
minimally proficient before beginning graduate medical
education.
34
As seen in Table 4.3, the items in the CEPAERs
list align well with the RIME framework and allow the
teacher to use prior knowledge of RIME to remember and
use the longer list of newer terms. In a report of 10 pilot
schools using EPAs to establish readiness for GME, 66% of
students met reporter EPAs, 36% met interpreter EPAs, and
17% met manager/educator EPAs.35 A similar
correspondence exists between the six ACGME competences
and, for instance, the 22 subcompetencies in internal
medicine26 and may also allow teachers to manage the more
granular system of competencies (see Appendix 4.4).
Descriptive Terminology for
Narratives in Evaluation
Why is it necessary to have a vocabulary of descriptive
evaluation for use by teachers in the clinical setting? Such
evaluations are often felt to be subjective and susceptible to
biases of the individual teacher.
36
Objective assessment tools,
such as multiple-choice tests or an objective structured
clinical examination (OSCE) using standardized patients,
have been considered to be more reliable. However, such
highly structured examinations are resource intensive, are
under the control of program and clerkship directors rather
than everyday teachers, and are difficult to arrange
frequently enough to provide ongoing feedback. Moreover,
they often assess only one dimension or competency at a
https://t.me/med1917
time. The ability to “put it all together” and bring it to the
current specific clinical situation usually requires an expert
to make the judgment; that is, a faculty member who is
trained or calibrated to make such determinations. In any
case, clinical teachers spend so much time with learners that
a descriptive vocabulary and framework are very helpful if
their observations are to be used for formative evaluation
(feedback) or summative evaluation (grading). We hope to
persuade our teachers that their descriptions of a student’s
behavior are not inevitably inferior to computerized tests
and high-fidelity simulations using mannequins; in fact,
because of its feasibility and easy application, we would
argue that “low-tech is good tech.”
The RIME scheme is one attempt to help teachers make their
observations more structured and more consistent, by
providing a useful description of what success looks like for
each trainee. In fact, using the RIME scheme it is possible to
achieve the level of reliability that is sufficient for pass/fail
decisions,
37
that has predictive validity for identifying poor
performance during internship,
38,39
and that helps achieve a
high degree of intersite consistency in a multisite
curriculum.40 In other words, evaluations using words can be
both reliable and valid, if they are part of a system of regular
frame of reference training for teachers (see Chapter 5).
41–43
It
may be more appropriate to refer to such evaluations by
teachers as descriptive, avoiding the term subjective, which
carries a pejorative connotation for those trained in sciences.
Teachers are more reluctant to offer comments on personal
behaviors, which they or students might consider hard to
measure (subjective); yet these are exactly what we must
https://t.me/med1917
capture if we are to give feedback on professional growth.
44,45
The RIME descriptive vocabulary has been reported to be
feasible and fair by students and faculty at multiple
institutions.
46–48
Perhaps more importantly, several studies
by Hemmer et al. have shown what would probably seem
intuitive: that teachers will tell you what they will not write
down on evaluation forms, and that this information is more
sensitive in detecting students who have deficiencies of
general knowledge on multiple-choice final examination
scores and in detecting students with professionalism
problems.
45,49
In other words, the “low-tech” method of
asking teachers what they think about students can be
helpful in providing students with interim information about
their progress that can help them anticipate summative
evaluations. The work of Hemmer also highlights the
importance of group process (i.e., asking faculty for an
evaluation using a framework like RIME) combined with the
evaluation forms to obtain a more complete picture of
performance (see Chapter 16).
The synthetic RIME framework provides a way for teachers
and trainees to visualize what success looks like. The RIME
terms are more concrete and are more behavioral than the
generic terms of the analytic models (knowledge, skills, and
attitudes) or the developmental model of Dreyfus (novice,
beginner, expert, etc., when used without narrative
descriptors). RIME takes advantage of clinicians’ ability to
make diagnoses from sets of observations and to classify
learners as reporting, interpreting, et cetera. Since its
“rhythm” (observation-reflection-action) parallels the day-today activities of clinicians and scientists, it has an intuitive
https://t.me/med1917
value and acceptance by teachers, as noted in Table 4.2.
There is probably not an intern in the United States who has
not written a “SOAP” note in which that classic rhythm of
observation-reflection-action is reproduced, with
observations recorded as “Subjective, Objective” and
reflection-action as “Assessment and Plan.” In other words,
the rhythm within the RIME scheme captures what
physicians and scientists do every day. It is simple without
being simplistic.
Complementary Frameworks:
ACGME General Competencies and
RIME
It is helpful to explain the how frameworks can be
complementary and not mutually exclusive. Three of the
ACGME/ABMS general competencies are the traditional
knowledge, skills, and attitudes of the analytic approach and
are implicit, not explicit, in the RIME framework. In other
words, if after a night on call the resident can successfully
propose an evidence-based management plan that
incorporates patient preferences, it follows that they have the
needed medical knowledge and the reasoning, interpersonal,
and communication skills and have accepted professional
ownership of the need to do so. The ACGME term Patient
Care is the primary competency and essentially a synthetic
term that is encompassed in the four terms of the RIME
scheme. Systems-based Practice is contained in the term
https://t.me/med1917
Manager, and Practice-based Learning and Improvement is
an advanced form of being an Educator.
The approaches used to assess development also have
alignment across competency milestones and EPAs, as
shown in Table 4.6
.
Table 4.5
Entrustment in Medical School Education
Proposed UME scale
1. Not allowed to practice
a. Inadequate knowledge/skill (e.g., does not know
how to preserve sterile field); not allowed to observe
b. Adequate knowledge, some skill; allowed to
observe
2. Allowed to practice only under proactive, full
supervision
a. As coactivity with supervisor
b. With supervisor in room ready to step in as
needed
3. Allowed to practice only under reactive/on-demand
supervision
a. With supervisor immediately available, all
findings double-checked
b. With supervisor immediately available, key
findings double-checked
c. With supervisor distantly available (e.g., by
phone), findings reviewed
4. Allowed to practice EPA unsupervised
5. Allowed to supervise others in practice
https://t.me/med1917
From Chen C, Sjoukje van den Broek WE, ten Cate O. The
case for use of Entrustable Professional Activities in
undergraduate medical education. Acad Med. 2015 Apr;
90(4):431-436. doi:10.1097/ACM.0000000000000586.
EPA, Entrustable Professional Activity; UME, undergraduate
medical education.
Table 4.6
Milestone Levels Related to Dreyfus Stages and Expected
Behaviors
Milestone
Level
Dreyfus
Stage
Learner
Behavior
Transition
to
Practitioner
Level of
Supervision
1 Novice Doing what
is told, rules
driven
Intro to
clinical
practice
Observation,
no
entrustment
2 Advanced
beginner
ComprehensionGuided
clinical
practice
Act under
direct
supervision
3 CompetentApplication
to common
practice
Early
independence
Act under
indirect
supervision
4 Proficient Application
to
uncommon
practice
Full
unsupervised
practice
Clinical
oversight
5 Expert Experienced,
up-to-date
Aspirational
growth
Supervise
others
https://t.me/med1917
clinician
Frameworks: Concluding Thoughts
Frameworks are not inherently right or wrong. They are
mental constructs that reflect goals and help structure
instruction and assessment. Frameworks are useful in their
different ways to help teachers assess trainees’ progress
toward independence. Synthetic models are strongest in
structuring observations made of learners in their actual care
of patients (in vivo), since these involve complex tasks that
require multiple attributes. Analytic models are best at
looking at discrete tasks whether in the care of patients (in
vivo) or under testing conditions (in vitro).
We wish to stress two principles of assessment in the clinical
setting. First, the framework must be accepted by the teacher
rating the learner, which we discussed earlier as construct
alignment. If the framework appears arbitrary to teachers,
they will feel free to use their own intuitive and potentially
less effective frameworks. Second, the framework must be
applied consistently across teachers and across students;
otherwise, the process is capricious. We should not assume
that any form or framework is so intuitively valid and easy
to use that teachers will apply it with consistency. Therefore
there must be ongoing training and feedback about the
framework and the use of the rating scale (we will discuss
faculty development later in this chapter).
To some extent the integrity of the assessment process for
trainees depends on teachers using it and using it
https://t.me/med1917
consistently. Many educators understand that an evaluation
form, checklist, or other form is not an evaluation tool, but a
way of communicating expectations to the teacher, who is in
fact the instrument (“tool”) of evaluation. This in turn may
depend on its ease of use, its portability from one trainee or
location to another, and its ability to be remembered. While
the primary effect of a framework is to structure learning
and assessment, its secondary effects involve the feasibility
of implementation and the faculty development resources
needed for use across a large faculty.
50
Our strategy is that
simplicity leads to acceptance; acceptance leads to use; use
leads to consistency; and consistency is an important element
of fairness.
Rating Scales
Rating scales arose from a need to evaluate areas of
performance not captured by standard knowledge-based
instruments, such as multiple-choice tests. Such scales are
still used now in the form of developmental Milestones for
assessment in GME and of entrustment scales for the abilities
of residents and students. This section will review both
classic and newer rating scales, with an emphasis on the
latter. It should be understood that while a scale tries to
distinguish levels of ability in a way corresponding to
arithmetical, ordinal integers, there is a continuum of ability
not just across learners but within a learner across patients
seen even in the same month of training and in how the
observations documented on the scale are used for formative
or summative purposes or for both.
51
Furthermore, by
https://t.me/med1917
definition all rating scales of human performance are ordinal,
meaning the “distance” between interval on the scale does
not necessarily equate to a linear, equal gain (or loss) in
ability.
Impetus for the development of rating scales came from two
sources in the early 1900s: psychologists looking to measure
human attitudes, and the Armed Forces who wanted to
better evaluate their trainees who were using new
technologies.
3
,5
In 1932 Likert developed the well-recognized
scale employing equal intervals and adding descriptors at
each point along the scale (i.e., Strongly Agree, Agree,
Undecided, Disagree, Strongly Disagree). Over the past 70
years many scales have been developed with better
psychometric properties, including specific rating scales and
evaluation forms for medical education. These forms were
developed with the goal of evaluating such important
competencies as clinical skills, clinical judgment and
decision-making, interpersonal and communication skills,
and professionalism.
Rating Scales: Basic Design
Most evaluation forms in medicine training currently use a
behaviorally anchored rating scale (BARS). A BARS form
provides descriptors of performance at various points along
the scale. The major evolution of BARS in medical education
has been the transition to greater use of developmental
behavioral descriptors like the competency milestones and
RIME synthetic framework. Older forms typically only
provided brief adjectival descriptors at the terminal ends of
https://t.me/med1917
Соседние файлы в папке Библиотека им академика М.И. Перельмана