Добавил:
Sekretar
kiopkiopkiop18@yandex.ru
t.me/Prokururor I Вовсе не секретарь, но почту проверяю
Опубликованный материал нарушает ваши авторские права? Сообщите нам.
Вуз:
Предмет:
Файл:Ординатура / Хирургия / Библиотека им академика М.И. Перельмана / Книга_2823_Библиотеки_им_академика_М_И_Перельмана
.pdf
FIG. 1.3 Logic model for assessment. Courtesy Elaine Van
Melle.
Accountability and Quality Assurance
The movement to CBME has been accompanied by
significant efforts to enhance the accountability of
physicians.3 Motivated by the need to improve quality and
safety—and in part by high-profile cases of derailed
physicians in the 1990s such as Michael Swango (USA) and
Harold Shipman (UK), and more recently neurosurgeon
Christopher Duntsch (USA)—the public has continued to
pressure medicine to increase its level of oversight and
eliminate the “bad apples.”
56,57
Medical educators are also
more keenly aware that too many trainees graduate with
substantial deficiencies in foundational knowledge as well as
clinical skills and other core competencies (e.g.,
professionalism, interpersonal skills, and communication)
important to succeed in our healthcare system
58–60
(see
Chapter 5). Effective quality assurance of promotion
https://t.me/med1917
decisions depends on robust assessment programs and is
critically important to ensure that graduates of medical
education programs are truly ready for the next stage of
training and, ultimately, unsupervised practice. Promoting
trainees who lack competence erodes, if not destroys, the
trust between the medical profession and the public. When
the focus shifts from promotion based on time-in-training to
promotion based on competence, adequate assessment
become even more key.
61
Quality Improvement Movement
At the same time, a variety of efforts have focused on
continuously improving the quality of healthcare. The
urgency to improve healthcare and achieve the Quadruple
Aim has only accelerated because of the COVID-19
pandemic.
62–64
These efforts have relied on methods devised
by workers in the field of quality improvement science and
engineering and, in some cases, have been used successfully
in industry for over 60 years to drive continuous
improvement in healthcare and are now increasingly being
used in medical education programs. Central to quality
improvement is assessment—it is very hard to improve
without meaningful measurement and data. Assessment offers
a means of identifying those whose overall performance is
well below expectations and identifying areas for
improvement for all learners in the medical education
system, helping to drive the continuous quality improvement
process. These developments have helped fuel the creation of
multiple new methods of assessment and increase the use of
other methods already available. For example, the
competency milestones, first implemented in the United
https://t.me/med1917
States in 2014, describe competencies in narrative,
developmental terms and use the principles of continuous
quality improvement as part of their foundation to improve
GME. The Milestones initiative can be viewed through the
lens of “action- or practice-based research” to learn and
develop evidence over time.
65
,66
There is no single “holy
grail” of assessment. All assessments have strengths and
weaknesses, and programs need to build in ongoing
evaluation of their assessment activities (see Chapter 18
).
Technology
Over the past 60 years, the availability of increasingly
sophisticated technology has changed the testing of medical
knowledge and judgment in fundamental ways.
67,68
The
introduction of the computer heralded an era of large-scale
testing by encouraging the use of multiple-choice questions
(MCQs), the answers to which could be scanned by machine,
turned into scores, and then reported in an efficient and
objective fashion.
More recently, the intelligence of the computer has
improved assessment in important ways:
1. It has enabled the application of significant
psychometric advances to the assessment of medical
knowledge. Specifically, the computer’s intelligence has
improved efficiency by allowing the selection of
questions that are targeted to the ability of particular
examinees. Sequential testing and adaptive testing
permit gains in efficiency and precision.
2. It has improved the assessment of higher cognitive
abilities, including clinical reasoning, by permitting the
use of interactive item formats that more closely
https://t.me/med1917
simulate the types of judgments physicians need to
make in practice (see Chapters 7
and 8).
3. It has enabled new approaches to assessment that
leverage the science of retrieval practice to enhance
long-term retention, such as progress testing and
confidence-based learning platforms.
69
While the impact of technology on assessment of clinical
skills has been slower to develop, advances in simulation and
computer technology have led to the development of
approaches and tools that re-create aspects of the clinical
encounter with considerable fidelity. These methods have a
growing impact on assessment, especially in the area of
procedural skills, where mastery models are beginning to
gain traction and have been shown to translate to the
bedside.
70–74
Finally, technology, especially through smartphone and
tablet applications, is beginning to change the way
assessment data is obtained and processed. For example,
tools designed for assessment through direct observation are
increasingly being converted into smartphone applications
75
(see Chapters 5 and 9). Natural language processing (NLP) is
enabling more feasible collection of narrative assessments,
and NLP software shows substantial promise in analyzing
narrative assessments for levels of developmental bias.
Learning management systems, increasingly used by
programs, are also beginning to incorporate mobile apps into
their platforms.75 These portable applications hold
substantial promise to reduce data collection burden while
guiding the assessment activity of the faculty to attend to
critical competencies.
https://t.me/med1917
Psychometrics
While the technology has improved, there have been
significant advances in psychometrics, the basic science of
assessment. Classical test theory, prominent from the mid20th century, has gradually given way to measurement
models based on strong assumptions about test items and
examinees. The family of item response theory models now
makes it possible to produce equivalent scores even when
examinees take tests made up of different questions.76 They
also support the computer-based administration of
examinations that are tailored to the ability level of
individual test-takers; this allows tests to be shortened by as
much as 40%.77 The ability to shorten tests has cost and
validity implications; less test material exposure decreases
the likelihood that future examinees are familiar with
examination content.78 Generalizability theory makes it
possible to help identify how much error is associated with
different facets of measurement (e.g., raters, patients).
79
Based on this information, assessments can be prospectively
designed to make the best use of resources, such as faculty
time, while maintaining the reliability of the results.
In addition to these major developments, there have been
several other advances. For example, a variety of systematic
methods are available for setting standards on tests and for
identifying when test questions are biased against particular
groups of examinees.
80,81
Test development methods have
gotten better, as has the means for judging whether particular
items are working properly. Overall, these advances have
improved both the quality and efficiency of assessment.
https://t.me/med1917
Qualitative Assessment and
Group Process
Although advances in psychometrics have clearly helped
improve assessment in medical education and will remain a
core science for assessment for the foreseeable future, many
have noted significant limitations of the traditional
psychometric approach in today’s complex clinical and
educational environment.82 Often referred to as qualitative
assessment or narrative assessment, use of the written word
has grown in importance. For example, many of the new
smartphone apps contain NLP capabilities that allow for the
capture of narrative assessment and feedback through
dictation. Milestones, discussed in more detail later in the
chapter, are more robust narrative descriptors of stages of
development, bringing both quantitative and qualitative
aspects of measurement closer together.74 Recent work has
also found that applying rigorous qualitative methodology to
assess narrative assessments can produce high levels of
reliability. Ginsburg and colleagues noted: “Using written
comments to discriminate between residents can be
extremely reliable even after only several reports are
collected. This suggests a way to identify residents early on
who may require attention. These findings contribute
evidence to support the validity argument for using
qualitative data for assessment.” They also noted that
reliability coefficients above 0.8 are possible with narrative
assessment.
83
Group process, commonly conducted by entities called
clinical competency committees (CCCs), has also become an
important and established part of assessment process and
https://t.me/med1917
programs (see Chapter 16). Effective group process can lead
to better judgments around competence, especially needed
for summative decisions.
84–88
Finally, qualitative research
techniques have been shown to have value in judging
aggregate assessment information, such as that contained
within a portfolio (see Chapter 15
). Again, a rigorous
approach to application of qualitative research techniques
and principles helps enhance the reliability and validity of
judgments.
89–91
Framework for Assessment
As methods of assessment have proliferated, so has the need
to use them efficiently and effectively as an integrated
combination in a system of assessment. Developing,
implementing, and sustaining effective systems for the
assessment of clinical competence in medical school,
residency, fellowship, and all graduate and postgraduate
health professions educational programs requires
consideration of what competencies need to be assessed, how
best to assess them, and the developmental level of the
trainee being assessed. Consequently, a three-dimensional
framework for structuring an assessment system can help
medical educators make better judgments about learner
development. Along the first dimension are the competencies
(i.e., abilities) that need to be assessed, the second is the type
of assessment required, and the third is the trainees’ stage of
development.
Dimension 1: Competencies
As shown in Table 1.3, there are several schemes for
https://t.me/med1917
describing the knowledge, skills, and attributes of the
physician.
24–27
The Canadian Medical Education Directives
for Specialists (CanMEDS) model, developed and
periodically updated by the RCPSC, describes the
competencies in terms of the roles of a physician and was last
updated in 2015 with plans to again review and revise in
2025. Good Medical Practice, which was created by the
General Medical Council in the United Kingdom, describes
the elements of good practice and was revised in 2019 to
highlight the problem of discrimination. In the United States,
two influential groups developed a set of core competencies.
The ACGME and the ABMS adopted six general
competencies in 2001 and have recently revised and updated
these competencies as subcompetencies within the
Milestones framework. These competencies comprise the
educational outcomes framework for residency and
fellowship training as well as maintenance of certification
programs throughout a physician’s career in the United
States. The NAM (formerly the Institute of Medicine)
recommended five core skills, or competencies, that create a
framework for evaluating performance and stimulating the
reform of education. CanMEDS, the ACGME/ABMS General
Competencies, and Good Medical Practice also highlight how
the competency frameworks evolve with time and changing
science as all three frameworks are now structured with
subcompetencies that reflect the multitude of abilities needed
and complexities of being a physician. They are intended to
improve professional education and practice with a goal of
enhancing the safety and quality of healthcare. Although
there are some differences among the schemes, there is also
significant overlap in these descriptions of a physician (Table
https://t.me/med1917
1.3).
Table 1.3
The Competencies and Subcompetencies of Physicians as
Described by Four Organizations
CanMEDS (Canada)
Competency
Domain
Subcompetencies
Medical expert
1. Practice medicine within their defined scope of
practice and expertise.
2. Perform a patient-centered clinical assessment and
establish a management plan.
3. Plan and perform procedures and therapies for the
purpose of assessment and/or management.
4. Establish plans for ongoing care and, when
appropriate, timely consultation.
5. Actively contribute, as an individual and as a
member of a team providing care, to the continuous
improvement of healthcare quality and patient safety.
Communicator
1. Establish professional therapeutic relationships with
patients and their families.
2. Elicit and synthesize accurate and relevant
information, incorporating the perspectives of patients
and their families.
3. Share healthcare information and plans with
patients and their families.
4. Engage patients and their families in developing
plans that reflect the patient’s healthcare needs and
goals.
5. Document and share written and electronic
information about the medical encounter to optimize
clinical decision-making, patient safety,
confidentiality, and privacy.
Collaborator
1. Work effectively with physicians and other
colleagues in the healthcare professions.
2. Work with physicians and other colleagues in the
healthcare professions to promote understanding,
manage differences, and resolve conflicts.
3. Hand over the care of a patient to another healthcare
professional to facilitate continuity of safe patient care.
Leader
1. Contribute to the improvement of healthcare
delivery in teams, organizations, and systems.
2. Engage in the stewardship of healthcare resources.
3. Demonstrate leadership in professional practice.
4. Manage career planning, finances, and health
human resources in a practice.
https://t.me/med1917
Health
advocate
1. Respond to an individual patient’s health needs by
advocating with the patient within and beyond the
clinical environment.
2. Respond to the needs of the communities or
populations they serve by advocating with them for
system-level change in a socially accountable manner.
Scholar
1. Engage in the continuous enhancement of their
professional activities through ongoing learning.
2. Teach students, residents, the public, and other
healthcare professionals.
3. Integrate best available evidence into practice.
4. Contribute to the creation and dissemination of
knowledge and practices applicable to health.
Professional
1. Demonstrate a commitment to patients by applying
best practices and adhering to high ethical standards.
2. Demonstrate a commitment to society by
recognizing and responding to societal expectations in
healthcare.
3. Demonstrate a commitment to the profession by
adhering to standards and participating in physicianled regulation.
4. Demonstrate a commitment to physician health and
well-being to foster optimal patient care.
General Medical Council (United Kingdom)
Competency
Domain
Subcompetencies
Professionalism
in action
Domain 1 describes six principles and
expectation of all physicians.
Domain 1:
Knowledge,
skills, and
performance
1. Develop and maintain your professional
performance.
2. Apply knowledge and experience to practice.
3. Record your work clearly, accurately, and legibly.
Domain 2:
Safety and
quality
1. Contribute to and comply with systems to protect
patients.
2. Respond to risks to safety.
3. Risks posed by your health.
Domain 3:
Communication,
partnership,
and teamwork
1. Communicate effectively.
2. Work collaboratively with colleagues.
3. Teaching, training, supporting, and assessing.
4. Continuity and coordination of care.
5. Establish and maintain partnerships with patients.
Domain 4:
Maintaining
trust
1. Show respect for patients.
2. Treat patients and colleagues fairly and without
discrimination.
3. Act with honesty and integrity
a. Communicating information
https://t.me/med1917
Соседние файлы в папке Библиотека им академика М.И. Перельмана