Добавил:
Sekretar
kiopkiopkiop18@yandex.ru
t.me/Prokururor I Вовсе не секретарь, но почту проверяю
Опубликованный материал нарушает ваши авторские права? Сообщите нам.
Вуз:
Предмет:
Файл:Ординатура / Хирургия / Библиотека им академика М.И. Перельмана / Книга_155_библиотеки_им_акад_М_И_Перельмана
.pdf
9.5 Screening Tests 187
Fig. 9.2 Indirect standardization
Direct and indirect standardization methods often yield similar outcomes, but
direct standardization is generally preferred over the indirect method. The
preference for direct standardization stems from the fact that it applies the
age-specific rates of the study populations to a single standard population, ensuring
consistent weighting across age groups. Conversely, in indirect standardization, the
weighting applied to the standard age-specific rates is contingent upon the age
distribution of the study populations. Consequently, if there are substantial
variations in age distribution among the study populations, the resulting
standardized mortality ratios (SMRs) may be based on disparate weightings,
rendering them incomparable.
Another advantage of direct standardization lies in its transitive property, as it
employs a consistent denominator across all datasets. For instance, if direct
standardization is employed on three datasets (C, D, and E), and the compressed
mortality file (CMF) between C and D is 2, while that between D and E is 0.5, it
implies that the CMF between C and E can be derived as 2/0.5 = 4.
9.5 Screening Tests
At times, there is a need to assess the efficacy of a particular test. This involves
comparing its outcomes with those of a widely recognized benchmark test, often
referred to as the “gold standard”, to ascertain the level of concordance and
compatibility. The gold standard test typically delivers the most accurate and
reliable results and serves as the benchmark against which the performance of other
tests is measured. However, this gold standard test may be costlier, more
time-intensive, or less accessible to all individuals. Alternatively, it may involve a
combination of diagnostic procedures typically conducted in a hospital setting,
rendering it unsuitable for routine or widespread use in screening.
For instance, consider the scenario where students undergo a basic hearing
assessment during their initial year of schooling. Those who do not pass this initial
screening test are subjected to retesting or are referred to a specialized hearing
clinic for more extensive evaluations to determine the presence of an actual hearing
impairment. Let’s assume a school with a student population of 600, out of which
https://t.me/med1917
188 9 Health and Disease Measures
60 students indeed have a genuine hearing impairment. Among these, 50 students
and 40 students with normal hearing (possibly affected by temporary conditions
like a cold during the test) failed the school hearing test. The outcomes of this
hearing test are outlined in Table
9.2.
As depicted in Table 9.3, each student in the school can experience one of four
possible outcomes. A student with an actual hearing impairment may either be
identified as having a problem during the test, which is a true positive outcome
(group “a” in Table
9.3), or may incorrectly pass the test, erroneously suggesting
the absence of a problem, resulting in a false negative outcome (group “c”).
Similarly, a student without a hearing impairment may correctly pass the test and
be accurately diagnosed as not having a problem, representing a true negative
outcome (group “d”), or may be erroneously identified as having a problem and fail
the test, resulting in a false positive outcome (group “b”).
In assessing the accuracy of the school hearing test, we evaluate two key aspects:
its ability to correctly identify students with hearing problems and its accuracy in
classifying students with normal hearing.
Referring to Table 9.3, vertically, we observe that 83% (50 out of 60) of students
with hearing problems were correctly identified as having issues during the school
test (cell “a”), while 93% (500 out of 540) of students with normal hearing were
accurately classified as such (cell “d”). These measures are known as sensitivity and
specificity, respectively.
Examining the table horizontally, we find that 56% (50 out of 90) of those
diagnosed with a positive result actually had a hearing problem (cell “a”), and 98%
(500 out of 510) of those diagnosed with a negative result were correctly identified
as having normal hearing (cell “d”). These metrics are termed positive predictive
value (PPV) and negative predictive value (NPV), respectively.
Table 9.2 Results from a
school hearing test program
School hearing test True hearing status
Have problem Normal To t a l
Reject (positive test result) 50 40 90
Pass (negative test result) 10 500 510
Total 60 540 600
Table 9.3 Possible outcomes
for a screening test
Disease status
Positive Negative
Test result Positive True
Positive
(a)
False
Positive
(b)
Negative False
Negative
(c)
True
Negative
(d)
https://t.me/med1917
9.5 Screening Tests 189
Sensitivity indicates the test’s ability to detect true positive cases, calculated by
dividing the number of true positive results (cell “a”) by the total number of subjects
with the disease (sum of cells “a + c”). In this example, the sensitivity of the school
hearing test is 83%.
Specificity, on the other hand, denotes the test’s capacity to correctly identify
disease-free subjects, obtained by dividing the number of true negative results (cell
“d”) by the total number of subjects truly without the disease (sum of cells “b + d”).
The specificity of the school hearing test is 93%.
Positive predictive value (PPV) gauges the probability of correctness of a positive
test, indicating the proportion of patients with positive results correctly identified as
having the disease. It is computed by dividing the number of true positive results
(cell “a”) by the total number of subjects with a positive test result (sum of cells “a
+ b”). In this scenario, the PPV of the school hearing test is 56%.
Negative predictive value (NPV) assesses the probability of correctness of a
negative test, reflecting the proportion of patients with negative results accurately
identified as not having the disease. NPV is calculated by dividing the number of
true negative results (cell “d”) by the total number of subjects with a negative test
(sum of cells “c + d”). The NPV of the school hearing test is 98%.
Example 4 A sample of 1000 people has been examined at a communicable disease
clinic. 50 of them were diagnosed with Covid-19 and the rest were healthy and
asymptomatic. Then the same people have been sent for PCR testing, and the results
of the test and the experts’ opinions are given in Table
9.4.
Estimate sensitivity, specificity, positive predictive value, and negative predictive
value.
Solution:Inthis a = 44, b = 9, c = 6 and d = 941.
Sen =
a
a + c
=
44
44 + 6
= 0.880
Spe =
d
b + d
=
941
941 + 9
= 0.991
PPV =
a
a + b
=
44
44 + 9
= 0.830
Table 9.4 Results of clinical diagnosis and PCR test for 1000 people
Clinic
+ −
PCR + 44 9 53
− 6 941 947
50 950 1000
https://t.me/med1917
190 9 Health and Disease Measures
NPV =
d
c + d
=
941
941 + 6
= 0.994.
Screening tests can be expressed as conditional probabilities in the following
relationships:
Prevalence = P
(
disease
)
Sen = P
(
test +|disease
)
Spe =
(
test −|no disease
)
PPV = P(disease|test+)
NPV = P
(
no disease|test−
)
1 − Spe = P
(
test +|no disease
)
1 − Sen = P
(
test −|disease
)
.
Using the above relations, the following equations can be derived between positive
predictive value, negative predictive value, sensitivity, and specificity:
PPV =
Sen × Pr
(
Sen × Pr
)
+
(
1 − Spe
)
×
(
1 − Pre
)
NPV =
Spe ×
(
1 − Pre
)
Spe ×
(
1 − Pre
)
+ Pre ×
(
1 − Sen
)
.
The other two rates, called “false positive rate” and “false negative rate”, along
with screening tests are also used. When the patient’s true situation is positive and
the test result is negative, it is called a false negative. If the patient’s true situation
is negative and the test result is positive, it is called a false positive. Table
9.5 can
provide the concepts of these two rates more clearly.
Table 9.5 Concepts of true
and false positive and
negative
Name of rate Disease situation Test result
True positive + +
False positive - +
False negative + -
True negative − −
https://t.me/med1917
9.5 Screening Tests 191
9.5.1 Confidence Interval for Screening Tests
To estimate the confidence interval for sensitivity (Sen) and specificity (Spe), it is
necessary to calculate the standard error of each one. Given that sensitivity and
specificity are considered a kind of ratio (p), their standard error is equal to:
Se
Sen
=
p
(
1 − p
)
n
=
Sen
(
1 − Sen
)
a + c
Se
Spe
=
p
(
1 − p
)
n
=
Spe
(
1 − Spe
)
b + d
.
Hence, the confidence interval for sensitivity (Sen) is equal to:
CI =
Sen ± z
1−α
2
× Se
Sen
.
The confidence interval for specificity (Spe) is equal to:
CI =
Spe ± z
1−α
2
× Se
Spe
.
Similarly, the standard error and the confidence interval for positive predictive
value (PPV) are obtained from the following equation:
Se
PPV
=
p
(
1 − p
)
n
=
PPV
(
1 − PPV
)
a + b
CI =
PPV ± z
1−α
2
× Se
PPV
.
Also, the standard error and the confidence interval for negative predictive value
(NPV) are obtained from the following equation:
Se
NPV
=
p
(
1 − p
)
n
=
NPV
(
1 − NPV
)
c + d
CI =
NPV ± z
1−α
2
× Se
NPV
.
Example 5
In Example 4, the sensitivity was estimated to be 0.88 and the positive
predictive value was to be 0.83. In that example, a = 44, b = 9, c = 6 and d = 941.
Find the confidence interval for sensitivity and positive predictive value.
Solution: The standard error of sensitivity and positive predictive values are:
https://t.me/med1917
192 9 Health and Disease Measures
Se
Sen
=
Sen
(
1 − Sen
)
a + c
=
0.88
(
1 − 0.88
)
44 + 6
= 0.0460
Se
PPV
=
PPV
(
1 − PPV
)
a + b
=
0.83
(
1 − 0.83
)
44 + 9
= 0.0516.
The confidence intervals for them are:
CI =
Sen ± z
1−α
2
× Se
Sen
=
(
0.88 ± 1.96 × 0.0460
)
=
(
0.790 to 0.970
)
CI =
PPV ± z
1−α
2
× Se
PPV
=
(
0.83 ± 1.96 × 0.0516
)
= (0.729 to 0.931.
9.6 Cohen’s Kappa Coefficient
Cohen’s kappa coefficient (κ) serves as a statistic used to assess inter-rater reliability,
particularly when observations are categorized as a qualitative dichotomous variable.
It offers a stronger and more robust measure compared to a simple percent agreement
calculation, as it considers the probability of agreement occurring by chance. It’s
imperative to note that both raters evaluate the same set of subjects. Hence, Cohen’s
kappa measures the agreement between two raters who each classify a group of n
subjects into two mutually exclusive categories.
Tip
Cohen’s kappa is only valid for nominal categories. If used for ordinal and
metric categories, it ignores their order and considers them as nominal.
To calculate Cohen’s kappa coefficient, the results of two experts should be
presented in a table, such as Table 9.6:
Table 9.6 General contingency table for kappa coefficient
Expert1
Yes No
Expert 2 Yes a b r
1
No c d r
0
c
1
c
0
n
https://t.me/med1917
9.6 Cohen’s Kappa Coefficient 193
Tip
Kappa values from different studies may not be comparable as the value also
varies with prevalence, which is the number of subjects in categories relative
to the total.
where r
1
and r
0
are the sum of rows of positives and negatives and c
1
and c
0
are
the column sum of positives and negatives. Given the cells in the table above, we first
obtain the probability of the observed agreement or the observed relative agreement
p
o
and the theoretical probability of a chance agreement p
e
between the two raters:
p
e
=
(
r
1
× c
1
)
+
(
r
0
× c
0
)
n
2
p
o
=
a + d
n
.
Cohen’s kappa coefficient is calculated from the following equation:
κ =
p
o
− p
e
1 − p
e
.
The kappa value is not like a correlation between −1 and +1 and may take any
negative values. But only values 0 to 1 are important for interpretation and have
useful meaning. All negative values are interpreted such as zero, which means that
there is no agreement at all between the two experts. A value of 1 indicates the perfect
agreement of the two experts, while the negative and zero values indicate that there is
no agreement between the two experts at all. Therefore, the closer kappa coefficient
is to 1, the greater the agreement, and the closer it is to zero, the lower the agreement.
Table
9.7 is a good guide for interpreting Cohen’s Kappa coefficient.
Table 9.7 Guideline for
interpreting Cohen’s Kappa
coefficient
Cohen’s Kappa Interpretation
0.0–0.19 Poor agreement
0.20–0.39 Fair agreement
0.40–0.59 Moderate agreement
0.60–0.79 Good agreement
0.80–1.0 Very good agreement
https://t.me/med1917
194 9 Health and Disease Measures
Tips
The following simple and practical assumptions must be fulfilled for Cohen’s
Kappa:
•
Independence of subjects,
•
Independence of raters, observers, and laboratories from each other,
•
The rating categories to which individuals are assigned are mutually
exclusive and exhaustive.
The assumptions for Cohen’s Kappa, which are easily fulfilled in most practical
situations, are that the categories are nominal and only valid for nominal categories.
This measure ignores the order and value of ordinal and metric categories and
considers them nominal. The other assumptions of this statistic are: the
independence of subjects, the independence of rater, laboratories and measurement
methods from each other, and the rating categories are mutually exclusive and
exhaustive.
9.6.1 Confidence Interval for Cohen’s Kappa
The formula for a kappa confidence interval is:
κ − 1.96 × Se
κ
to κ + 1.96 × Se
κ
,
where is Se
κ
the standard error of kappa obtained by:
Se
κ
=
p
o
(
1 − p
o
)
n
(
1 − p
e
)
.
Example 6 A sample of 1000 people has been examined at a communicable disease
clinic. 50 of them were diagnosed with Covid-19 and the rest were healthy and
asymptomatic. Then the same people have been sent for PCR testing, which is the
result of the test and the experts’ opinions are given in Table
9.8.
Find kappa coefficient and its 95% confidence interval.
Solution: The given information in this example is a = 44, d = 941, r
1
= 53, r
0
=
947, c
1
= 50, c
0
= 950, and we substitute them in the formulas:
Table 9.8 Results of clinical diagnosis and PCR test for 1000 people
Clinic
+ −
PCR + 44 9 53
− 6 941 947
50 950 1000
https://t.me/med1917
9.6 Cohen’s Kappa Coefficient 195
p
e
=
(
50 × 53
)
+
(
947 × 950
)
1000
2
= 0.9023
p
o
=
44 + 941
1000
= 0.985
κ =
p
o
− p
e
1 − p
e
=
0.985 − 0.9023
1 − 0.9023
= 0.846.
The agreement between clinical diagnosis and PCR test is %84.6 and very good
and nearly perfect.
The standard error of kappa is:
Se
κ
=
p
o
(
1 − p
o
)
n
(
1 − p
e
)
=
0.985
(
1 − 0.985
)
1000
(
1 − 0.9023
)
= 0.0249.
The 95% confidence interval for Cohen’s kappa is
κ − 1.96 × Se
κ
to κ + 1.96 × Se
κ
= 0.846 − 1.96 × 0.0249 to 0.846 + 1.96 × 0.0249
=
(
0.797 to 0.895
)
.
If the two experts are in perfect agreement with each other, it means that in
Table
9.10,the b and c cells should be 0, i.e., the table should be like Table 9.9.
In case the two raters have negative agreement with each other at all, it means that
in Table
9.9, a and d cells should be 0, i.e., the table should be like Table 9.10.
Table 9.9 Contingency table for two experts with perfect agreement
Expert1
Yes No
Expert 2 Ye s a 0 a
No 0 d d
a d n
Table 9.10 Contingency table for two experts with negative agreement
Expert1
Yes No
Expert 2 Ye s 0 b b
No c 0 c
c b n
https://t.me/med1917
196 9 Health and Disease Measures
Example 7 Suppose that in Example 1, the result of clinical diagnosis and PCR test
is exactly the same, i.e., b = 0 and c = 0. Find the kappa coefficient.
Solution: If the clinical diagnosis and PCR test are exactly the same, we will have
Table
9.11.
The probability of observed agreement, agreement by chance and kappa
coefficient is:
p
e
=
(
50 × 50
)
+
(
950 × 950
)
1000
2
= 0.905
p
o
=
50 + 950
1000
= 1.00
κ =
p
o
− p
e
1 − p
e
=
1.00 − 0.905
1.00 − 0.905
= 1.00.
This kappa means that the two raters are in perfect agreement.
Example 8 If in Example 6, the result of clinical diagnosis and PCR test is not the
same at all, i.e., a = 0 and d = 0. Find the kappa coefficient.
Solution: If the clinical diagnosis and PCR test are exactly the same, we will have
Table
9.12.
The probability of observed agreement, agreement by chance, and kappa
coefficient is:
p
e
=
(
50 × 50
)
+
(
950 × 950
)
1000
2
= 0.905
Table 9.11 Results of clinical diagnosis and PCR test with perfect agreement
Expert1
Yes No
Expert 2 Yes 50 0 50
No 0 950 950
50 950 1000
Table 9.12 Results of clinical diagnosis and PCR test with no agreement at all
Expert1
Yes No
Expert 2 Yes 0 50 50
No 950 0 950
950 50 1000
https://t.me/med1917
Соседние файлы в папке Библиотека им академика М.И. Перельмана