Making Sense of Data
Task 1
In this exercise you are asked to consider ways of appraising the validity of a mea-sure. We will use a fictional example, to prevent you from being influenced by your prior knowledge about the measure. TV dementia is an imaginary common disease caused by excessive exposure to television. It is characterized by a long symptom-free period, followed by progressive mental deterioration and culminating in inability to perform activities of daily living unaided. Assume that the diagnosis can be determined with certainty, before or after the development of symptoms, by accurate but costly and elaborate tests. In a study using a new simple test, imaginatively named test A, the prevalence rate of the disease in a population was found to be 18.4 per 100.
Question 1-1
How could you appraise the validity of the test?
Question 1-2
What kinds of evidence would be helpful? Mention as many possibilities as you can.
Task 2
Question 2-1
The validity of test A was measured by applying it to 100 patients known to have TV dementia and 400 people known to be free of this disease; there were 80 positive results in the first group, and eight in the second. What are the sensitivity and specificity of the test, and what are the false negative and false positive rates?
Table 1. Test Results in a Sample of Diseased People
|
Test Result |
Number |
|
Positive |
a |
|
Negative |
b |
|
Total |
a + b |
Table 2. Test Results in a Sample of Disease-Free People
|
Test Result |
Number |
|
Positive |
c |
|
Negative |
d |
|
Total |
c + d |
Question 2-2
Is there anything else you would like to know before using these findings?
Question 2-3
If a measure used for determining the prevalence of an attribute has a low sensitivity, how will this affect the prevalence rate?
Question 2-4
If the measure has a low specificity, how will this affect the prevalence rate?
Question 2-5
Can you calculate the prevalence rates that test A will yield in populations (Pepi and Quepi) where the true prevalence rates are 21% and 7%, respectively. If this is too complicated, just guess.
Question 2-6
According to the true prevalence rates in Pepi and Quepi, the rate ratio is 3. If we used the prevalence rates yielded by test A, do you think the rate ratio would
be the same, lower, or higher?
Task 3
Question 3-1
Dissatisfied with test A, Dr. B has developed a new test for TV dementia. This test, named test B after its inventor, has a sensitivity of 99% and a specificity of86%. Test B is now used to measure the prevalence of the disease in Quepi, and the result is compared with the rate (using test A) in Pepi; the latter rate, you will remember, was 18.4%, and the true prevalence rate in Pepi was three times that in Quepi. Without doing any calculations, can you say whether the ratio of the rate in Pepi (using test A) to the rate in Quepi (using test B) will be more than 3, between 1 and 3, or less than 1?
Question 3—2
Construct and fill in a table (like Table 3-) to show the expected results when Test B is used in Quepi. You can then supply the rate ratio requested in Question 3-1.
Table 3
|
|
Disease |
|
|
|
Test Result |
Absent |
Present |
Total |
|
Positive
|
|
|
|
|
Negative |
|
|
|
|
Total |
|
|
|
Task 4
In which of the following studies would you suspect that an observed association might be an artifact (or spuriously strong) because of differential validity?
1. A comparison of the incidence of schizophrenia in two countries, based on the diagnoses recorded in clinical files by psychiatrists.
2. A study of the association of retinal disease with diabetes, based on the clinical records of people with and without diabetes.
3. A study of the efficacy of immunization against a specific disease, based on a comparison of the subsequent incidence of the disease in volunteers who were immunized and in people who were not immunized.
4. A study of the efficacy of a new treatment for painful menstruation, in which the proponents of this treatment questioned patients about the persistence of their symptoms, after randomly dividing them into two groups—-one whose members received the new treatment (without their knowledge) and one whose members continued their usual treatment.
5. A study of the relationship between exposure to anesthetic gases and a specific immunodeficiency disorder, using a test (for the disorder) with a specificity of 100% but a sensitivity of only 60%.
6. A study of the association of senile dementia with educational level, using simple tests of cognitive functioning (general knowledge and intellectual capacity) to measure senile dementia.
7. A study of the association between fever in early pregnancy and congenital anomalies, in which mothers of deformed and normal babies were questioned about the illnesses they had had during their pregnancy.
8. A study of the effect of smoking on physical fitness, in which smokers were compared with people who had given up smoking.
9. A study of the effectiveness of an intensive educational program on hygienic practices, in which school children who had been exposed to the program were asked whether they washed their hands before eating, and their replies were compared with those of similar children who had not been exposed to this program.
10. A study to determine whether rheumatoid arthritis "runs in families," in which patients with this disease and controls who were free of it were asked whether their parents had arthritis.
11. A study of the association between respiratory disease and disease of the locomotor system (bones, joints and muscles), based on an analysis of the diagnoses recorded in hospital patients.
12. A study of international variations in the prevalence of gallstones, based on the crude findings of all autopsy studies published since 1890 (Brett and Barker, 1976).
Task 5
Question 5-1
There are two tests for the detection of TV dementia—test A (sensitivity 80%, specificity 98%) and test B (sensitivity 99%, specificity86%). Which would be a better screening test, and why?
Question 5-2
What other information (besides sensitivity and specificity) would be helpful in appraising the value of a screening test?