Добавил:
Sekretar
kiopkiopkiop18@yandex.ru
t.me/Prokururor I Вовсе не секретарь, но почту проверяю
Опубликованный материал нарушает ваши авторские права? Сообщите нам.
Вуз:
Предмет:
Файл:Ординатура / Хирургия / Библиотека им академика М.И. Перельмана / Книга_2823_Библиотеки_им_академика_М_И_Перельмана
.pdf
the scale and we discourage their use moving forward. (An
example of a discontinued older version is shown in
Appendix 4.3.)
It is becoming more apparent that the choice of scale anchors
is important. Box 4.1
describes various types of anchors for
scales. These are not mutually exclusive, but when choosing
a scale several considerations are paramount:
1. Returning to the earlier part of the chapter, the scale
anchors must align with the purpose of the assessment
and the framework chosen. “Quality” anchors such as
superior, excellent, and unsatisfactory align poorly with
developmental and synthetic frameworks. Even when
using an analytic framework (see Fig. 4.2A) these type of
anchors require the additional work of translation on the
part of the rater. In essence, these types of scales represent
a strange form of encryption that is often not accessible to
the rater or the learners. Evaluation forms and scales, as
noted earlier, signal what is important and should guide
judgment. When additional, nebulous steps are added
some translation is required and ineffective use of the
scales results. Chapter 5 covers in greater detail issues
around the frame of reference faculty use when providing
ratings.
2. Scales have to be aligned with the assessment task and
purpose.
3. Faculty development is absolutely crucial. Evaluation
forms are not the measurement instrument; the rater is the
true instrument and they need training to develop shared
mental models around the evaluation framework, task,
and purpose (see Chapter 5).
https://t.me/med1917
4. Where possible, criterion-referenced scales and forms
(such as RIME and the Milestones) are preferable.
Box 4.1
Various Types of Anchors for Scales, Both Preferred and
Cautiously Usable
Rating scales: Types of anchors
A. Still in use but should be used cautiously based on
assessment purpose:
•. Performance “quality”
•. E.g., unsatisfactory-satisfactory-superior
•. Frequency
•. Rarely–always
•. Normative
•. Level of comparative performance (e.g., peers; stage of
training, etc.)
B. Preferred and recommended whenever possible:
•. Developmental
•. Entrustment/supervision
•. Narrative
These can overlap depending on purpose and construction of
the scales. For example the Internal Medicine
subcompetencies include a developmental scale (levels)
where Level 4 is designated as “Ready for Unsupervised
Practice” (entrustment/supervision) using narrative
(Milestones as a format of a behaviorally anchored rating
https://t.me/med1917
scale) to describe each level.
Appendix 4.2 shows an example of a BARS form for medical
students from the Uniformed Services University of the
Health Sciences containing detailed descriptors
(incorporating “RIME” terms) at each level of performance
on a 5-point scale. This is a better example of a constructaligned scale.
The optimal range of a numeric scale is debated, but most
experts recommend a scale contain between four and nine
gradations depending on purpose, and these ranges have
been retained in both Milestones and EPAs. A 9-point scale
can be helpful when comparing a large population of
trainees. The best example is the validity studies of the US
Milestones that use a five-level developmental scale with
transition zones between levels, resulting in a 9-point scale.
As seen in Fig. 4.6, the Internal Medicine Milestone for taking
a Medical History within the Patient Care Milestone, there
are five levels of ability but the 9-point scale is retained.
More importantly, the behavioral anchors in the BARS form
provide verbal descriptions to be used by the teacher in
classifying a resident’s ability along the entire scale.
https://t.me/med1917
Fig. 4.6 Internal Medicine Milestone Patient Care 1:
History. There are transitional zones between levels.
As stated before, the Milestones of most specialties use a 5level system in which 1 = novice or advanced beginner and
5 = “aspirational” for a trainee (but might be expected of
someone in practice). One specialty, general surgery, decided
to label the first level “critical deficiencies” to signify
significant deficiencies needing urgent intervention. In Fig.
4.6, the 5 levels show a level of progression that is similar to
the levels of entrustment in EPAs (Table 4.5). One very
important caveat should be noted about the Milestones used
in the United States. Milestones are designed to guide the
judgment of clinical competency committees (CCCs)
reviewing 4 to 6 months of resident performance and
assessment data (discussed later in the chapter) and are not
recommended for use as an evaluation form for short rotations.
However, some GME programs have used the Milestones in
their specialty as an “item bank” to construct more focused
and meaningful assessment forms for specific curricular
experiences. Much more work is needed in this area, but
https://t.me/med1917
some early work suggests this may be a useful approach for
programs.
52–54
Table 4.7
Examples of Narrative Text for the Levels of the Patient Care
History Milestone for Internal Medicine
Patient Care 1:
History
Overall intent:
To competently
interact with
patients from
diverse
backgrounds
and consistently
use all available
resources to
obtain a
comprehensive
patient history
Milestones Examples
Level 1 Elicits
and reports a
comprehensive
history for
common patient
presentations,
with guidance
• Obtains accurate, patient-centered
history from a 30-year-old patient with
a red swollen joint using open-ended
and directed questions but without
exploring clear underlying hypotheses
• Presents oral and written report that is
organized but not focused on the chief
https://t.me/med1917
Seeks data from
secondary
sources, with
guidance
complaint
• Needs prompting to seek data from
family members, ancillary staff
members, outside pharmacy, outside
labs, and databases for controlled
substances
Level 2 Elicits
and concisely
reports a
hypothesis-driven
patient history for
common patient
presentations
Independently
obtains data from
secondary sources
• Interviews a patient with no past
medical history with a chief complaint
of a red swollen joint, asking the patient
about recent alcohol use, diet, trauma,
sexual history, and other pertinent
questions; reports history limited to
pertinent positive and negative facts
• Respectfully uses the pronouns that a
transgender patient identifies with and
asks pertinent sexual orientation and
activity questions to provide highquality care in primary care clinic
• Without prompting, reviews and
presents relevant data from previous
medical records, including past labs and
primary care physician notes, family
members, ancillary staff members,
outside pharmacy, outside labs, and
databases for controlled substances
• Proactively reviews prescription
history from available databases and
calls the patient’s pharmacy for recent
prescriptions that note allopurinol has
not been refilled in months
Level 3 Elicits • Presents an 85-year-old with a history
https://t.me/med1917
and concisely
reports a
hypothesis-driven
patient history for
complex patient
presentations
Reconciles
current data with
secondary sources
of congestive heart failure, coronary
artery disease, chronic obstructive
pulmonary disease, and diabetes with a
chief complaint of several weeks of
shortness of breath, asking about
medication and dietary adherence;
reports on the presence of angina or
heart failure symptoms, recent upper
respiratory infection, and allergen
exposure
• Completes accurate medication
reconciliation using multiple sources
and clarifies history based on new
information as it becomes available
from caregivers who note recent weight
gain
Level 4
Efficiently elicits
and concisely
reports a patient
history,
incorporating
pertinent
psychosocial and
other
determinants of
health
Uses history and
secondary data to
guide the need for
(Note: Example uses same patient from
Level 3)
• Discovers the patient has not
filled recent prescriptions and
determines it was due to an
insurance lapse, and that the
patient does not have reliable
transportation to a pharmacy
• Determines patient has no
reliable prescription plan
coverage
• Determines patient recently had
cardiac work-up at another
hospital 1 month ago and does
https://t.me/med1917
further diagnostic
testing
not order echocardiogram based
on previous results
Level 5
Efficiently and
effectively tailors
the history
taking, including
relevant historical
subtleties, based
on patient,
family, and
system needs
Models effective
use of history to
guide the need for
further diagnostic
testing
• Obtains a history from a patient
presenting with macrocytosis, gout, and
liver function test abnormalities,
building trust to explore relevant
history and learns that the patient
consumes alcohol despite initial denial
• Takes a history from an injured
patient and realizes that the boyfriend
answers all the questions; identifies that
the patient may be a victim of intimate
partner violence based on nonverbal
cues
• Obtains history of medication
prescription plan and recognizes that
patient may not have Medicare Part D
or is in the coverage gap (i.e., “donut
hole”)
• Evaluates a patient with a complaint
of headache and illustrates to the more
junior learners the elements of the
history that preclude the need for
additional testing
Purposes and Advantages of
Evaluation Forms
Relative to other evaluation tools, evaluation forms can be
https://t.me/med1917
relatively time efficient for the program or clerkship director.
To supplement forms using Milestones, programs can
modify or develop evaluation forms to suit specific needs.
However, several additional caveats should be noted if you
choose to develop a new evaluation form with a rating scale.
First, the training program should assess, at a minimum, the
reliability if not also the validity of the forms (see Chapter 2
).
Second, development of “new” forms, independent of efforts
to teach faculty how to effectively use the new forms, does
not necessarily lead to more reliable or valid assessments of
the resident.
55
In fact, attention has shifted away from
developing “better” forms because most performance
appraisal experts believe that more focus is needed on how
to train raters to use the form more effectively.
56,57
This is
why awareness of the evaluation framework discussed
earlier is so important,
45
and we will provide suggestions for
faculty training later in the chapter (also see Chapter 5
). We
should emphasize that reliability can be increased by
increasing the number of observers and/or the observations
in a composite evaluation (see Chapters 2 and 5).
Evaluation forms, if used consistently by teachers, can
provide a longitudinal “composite” assessment. Other tools,
such as standardized patients, while very valuable, usually
only provide a cross-sectional assessment at a single point in
time. Evaluation forms have the potential to prompt and
document judgments of individual faculty, based on
multiple observations, conducted over time. The first task of
rating forms should be to structure the observations of
faculty so that their “findings” (analogous to a patient’s
symptoms or vital signs) are focused to the goals of the
https://t.me/med1917
program and are not idiosyncratic to the observer.
Observations must then be interpreted and placed in an
evidence-based framework being used by the program (e.g.,
RIME, Milestones, and/or EPAs), and a conclusion reached
as to whether this learner is meeting the expectations or
values of the program and profession; this interpretation of
the observations is called evaluation. After the teacher has
observed one or a few interactions with patients the
evaluation yields information that may be used to provide
feedback to the learner about their progress. However,
conversion of the evaluations into a grade or an
advancement decision is an administrative action rather than
simply an educational one.14 This conversion of information
into knowledge requires more certainty achieved through
sufficient and reliable sampling and documentation of
information across enough observers and important skills
(e.g., information gathering, clinical reasoning, procedures)
across a variety of clinical problems (trauma, infection,
cardiac, etc.) in the settings in which the learner will practice
(e.g., outpatient, emergency, inpatient). It also requires that
the decision-making group (promotions committee or
competency committee) understand the uses and limitations
of the rating forms and assessment techniques that provide
the material for a decision.
Evaluation forms can also help minimize the potential bias of
the Hawthorne effect: when the process of measurement
itself affects what is being measured by collecting
observations and judgments over a period of time. However,
the impact of the Hawthorne effect is actually quite modest
and tends to minimize with habituations over time.
https://t.me/med1917
Соседние файлы в папке Библиотека им академика М.И. Перельмана