Mean of two visits (D)

Fig. 1.4 A Bland–Altman plot of test-retest differences showing the repeatability of subjective refraction. (Reprinted from Sheedy et al. 2004, with permission of The American Academy of Optometry)

correlation coefficient found for unaided acuity is in part due to the much larger range of values used in its calculation (6/60, 20/200 to 6/4, 20/13), compared with the calculation involving aided acuity (6/6, 20/20 to 6/4, 20/13) (Elliott & Sheridan 1988). Correlation coefficients can be used when comparing tests that do not use the same units, but their limitations need to be realised. In particular, a large range of values should be used, so that correlation coefficients are not artificially low. Concordance values (the percentage of patients getting exactly the same score on test and retest) have also been used to indicate that a test is repeatable. However, a high proportion of patients often obtaining exactly the same score on follow-up visits indicates that the step sizes on the test are too big rather than that the test is repeatable (Bailey et al. 1991). For example, a visual acuity chart containing only 20/20 (6/6) and 20/200 (6/60) lines would provide very high concordance but would be of very little value.

Coefficients of repeatability results are also of value because they indicate the size of a clinically significant change. Repeatability studies providing COR data indicate the size of the change in score due to chance. A significant change in score would then be anything larger than the COR (at least for tests with a continuous scale). More studies are required that indicate the size of a clinically significant change for optometric tests. Repeatability appears to be the most important quality of a test, as it influences the others. For example, if a test has poor repeatability and test results correlate poorly with retest results, it is unlikely that results from

the test correlate highly with a gold standard measure. Therefore, its validity is poor. If a test correlates poorly with itself, how can it correlate well with something else? Similarly, a test with poor repeatability is likely to have a large range of scores in patients with normal, healthy eyes. For an abnormal eye to be outside this large normal range would be more difficult, so good discriminative ability is less likely. Optical companies should be encouraged to provide repeatability data with any newly available test, so that it can be compared with the currently available tests.

1.2 ROUTINE SCREENING

Another area of the primary eye care examination that has been discussed in detail in the research literature is the issue of using tests for screening purposes. This issue has become more pertinent to optometry with the increasing use of clinical assistants to routinely perform various automated or simple tests as part of a screening process prior to the primary eye care examination.

1.2.1 The use of clinical assistants in routine screening

The rationale behind the use of clinical assistants in pre-examination screening is twofold:

■These procedures generally become more routinely performed.

■As clinical assistants perform certain tests that the optometrist would previously have performed, some of the optometrist’s time is freed up. They could use this time to perform additional procedures or examine more patients per day. The pros and cons of clinical assistants performing ocular health screening tests routinely are further discussed in the following sections.

What procedures and tests can a clinical assistant perform? After a period of training, they should be able to competently perform any automated procedure, such as automated visual fields and focimetry, autorefraction and non-contact tonometry. In addition, other simple tests could be performed such as colour vision and stereopsis

screening and interpupillary distance (PD) measurement. It is not possible for a clinical assistant to complete the full case history, since history taking continues throughout the examination. However, assistants could record a baseline history that could be reviewed and augmented by the clinician. However, this approach provides less likelihood of a good rapport being established between patient and clinician, which is vital for an optimal examination result (Ettinger 1994). Clinical assistants could also measure visual acuity with the patient’s spectacles. However, important information can be obtained during visual acuity measurement in addition to the acuity score and, as an important part of the subjective refraction is to compare the final visual acuity (which the optometrist measures) with the habitual acuity, it appears best to have both measurements made by the clinician.

1.2.2 False positives and the positive predictive value

The decision to use any screening test/procedure routinely is more complicated than boldly stating ‘I want to catch all patients with . . . glaucoma, a retinal detachment, a brain tumour, etc. . . . so I am going to perform this test on everybody’. To understand the advantages and disadvantages of screening, it is important to understand sensitivity and specificity. How good do you think your screening tests are? If a test indicates that a patient has an eye disease, what are the chances that they actually have the condition? When considering this question, you must not only consider how good the test is at identifying the disease, but you must also consider how good the test is at correctly identifying someone as normal. Unfortunately, all tests provide false positives: patients with normal, healthy eyes which the test results suggest are abnormal. There

are four possible outcomes from the results of a screening test (Table 1.1) and this information is used to quantify how well the test discriminates between ‘normal’ and ‘abnormal’ eyes, by providing sensitivity and specificity values.

■Sensitivity is the ability of the test to identify the disease in those who have it.

■Sensitivity TP/(TP FN)

■Specificity is the ability of the test to correctly identify those who do not have the disease.

■Specificity TN/(TN FP)

■The false positive rate is simply 1 minus the specificity.

■Another important term to understand is the Positive Predictive Value ( PV). This is the proportion of people with a positive test result who have disease.

■PV TP/(TP FP)

The reported sensitivity and specificity of a test will differ depending on the pool of patients examined, the gold standard used to determine the presence or absence of disease and the cut-off criteria used.

The ability of a screening test to correctly identify patients with disease is highly dependent upon how prevalent the condition is (Bayes Theorem). For example, let us consider primary open-angle glaucoma (POAG), which has a prevalence in the over-40 population of approximately 1%, and assume that we have a screening test for glaucoma with 95% sensitivity and 95% specificity. Out of 1000 patients, 10 (1%) could have POAG and it is likely that 9 or all 10 would be detected, as the test sensitivity is 95%. With a false positive rate of 5% (1–0.95), the screening test would also suggest that about 50 of the remaining 990 patients with normal healthy eyes actually had POAG. In total, 60 patients would give positive results, of which only 10 would have the disease ( PV 17%). If the

sensitivity and specificity of the screening test was an amazing 99%, there would be 10 true positives and 10 false positives, and there is still a 50% chance that a patient with a positive result is normal.

Now let us consider the likely outcomes using the same test on patients with a family history of POAG where the prevalence of the disease is higher at about 10%. For the screening test with 95% sensitivity and specificity, out of 1000 patients, 100 would have POAG and it is likely that 95 would be detected. With a false positive rate of 5%, the screening test would also suggest that 45 of the remaining 900 patients with normal healthy eyes actually had POAG. In total, 140 patients gave positive results, of which 95 had the disease ( PV 68%). For the 99% sensitivity and specificity screening test, there would be 99 true positives and 9 false positives and a PV of 92%.

1.2.3 Are false positives so important?

Elmore et al. (1998) reported the false positive rate of breast cancer screening tests to be 6.5% and 3.7% respectively. These translate to very good specificity values of 93.5% and 96.3%, which are similar to the specificity of screening tests used in primary eye care (Bullimore 1998a). Despite this good specificity, over a ten-year period, nearly one-third of the women screened had at least one false positive mammogram or clinical breast examination. It has been shown that these false positive results have negative psychological effects on these women (Brett & Austoker 2001) and likely their families. Imagine being told that you or one of your family has received a positive screening test and has to undergo further tests for breast cancer.

Similarly, there is considerable and unnecessary worry and stress caused by a false positive result leading to referral to a secondary eye care system, in that some patients worry that they might be going blind. Patients should not be referred to secondary eye care on the basis of a slightly high intraocular pressure using a non-contact tonometer or a single positive visual field screening or because one clinician judged an optic disc to be suspicious (Bullimore 1998a). In addition to the psychological effects on patients and their families, the cost of all the subsequent clinical tests prompted by a positive screening result should also be considered.

1.2.4 Screening all patients

It is important to understand the limitations of screening. Realise that one positive screening result does not mean somebody has the disease being screened for and in many cases the chances that they have the disease remains small (Bullimore 1998a). Screening tests used to detect diseases with a low prevalence should have a reasonable specificity (typically 95%) and yet retain good sensitivity (typically 90%). However, even then, if a positive result is obtained, the test should be repeated. For example, as part of the ocular hypertension treatment study, Keltner et al. (2000) found 703 Humphrey visual field test results that showed abnormal (positive glaucoma hemifield test and/or Corrected Pattern Standard Deviation, p 0.05), and reliable visual fields. On retesting, abnormalities were not confirmed for 604 (86%)! The vast majority of visual field abnormalities were not verified on retest and confirmation of visual field abnormalities is essential for distinguishing reproducible visual field loss from long-term variability.

Because of false positive results, if you test a healthy eye often enough, it will sooner or later give a positive result. The poorer the specificity of a test, the more likely this is to happen. Elmore et al. (1998) estimated that the cumulative risk of having at least one false positive result after 10 mammogram screenings was nearly 50% ( 1–0.93510).

Similarly, when screening for POAG visual field defects every year for 10 years with a test with a specificity of 95%, the cumulative risk of having at least one false positive result is 40% ( 1–0.9510). Even if the test specificity is 99%, at least one annual assessment over a 10-year period would be a false positive.

1.2.5 Overcoming the problem of false positives: repeat tests

The best way to keep false positive referrals to a minimum is to repeat positive results. For example, the 95% sensitive and specific glaucoma screening test discussed in section 1.2.2 would produce positive results in 60 patients if 1000 were screened, yet only 10 would have the disease ( PV 17%). If all these 60 patients were retested, 9 or all 10 of the

glaucoma patients would be identified. However, 95% of the false positives (47 or 48) would now give a normal result. On retesting, positive results are found for 13 patients, of whom 10 have the disease ( PV 77%).

This approach is further improved by only screening those patients that are ‘at risk’. In these patients, the prevalence of the disease is much higher than in the general population. Let us consider the likely outcomes using the same tests on patients with a family history of POAG where the prevalence of the disease is about 10%. For example, the 95% sensitive and specific glaucoma screening test discussed in section 1.2.2 would produce positive results in 140 patients if 1000 were screened and 95 would have the disease ( PV 68%). On retesting the 140 patients, 90 of the glaucoma patients are detected. Of the 45 false positives, 43 of them now give a normal field. After the repeat test, 92 give positive results, of which 90 have glaucoma ( PV 98%). Of course, 5 glaucoma patients will now not be referred, so that a patient with a positive field test, even when followed by a normal field, should be closely monitored. The positive predictive value is also improved if you just perform screening on all patients over 75 years of age (prevalence of POAG 5%;PV 50%, after repeat testing, PV 96%) or patients over 40 years of age who are African American, diabetics or have high myopia or those with suspicious optic discs, high intraocular pressure, etc.

1.2.6 Routine fundus dilation?

There has been considerable debate about whether a primary care eye examination should routinely include a dilated fundus examination or DFE (Siegel et al. 1990, Parisi et al. 1996, Batchelder et al. 1997, Bullimore 1998b). Two main arguments, supported by clinical data, are proposed in favour of the DFE. The first is that a DFE increases the number of posterior pole anomalies detected (Siegel et al. 1990, Parisi et al. 1996). In these studies, a nondilated fundus examination with direct ophthalmoscopy was compared to a DFE using headband binocular indirect ophthalmoscopy (BIO) and direct ophthalmoscopy. Siegel et al. (1990) also used a monocular indirect ophthalmoscope examination as part of the non-dilated exam. The poor field of view of the direct ophthalmoscope was

particularly blamed for missing anomalies in the posterior pole as it is too small to examine the area quickly and easily. For further clarification of the need for a DFE to detect posterior pole anomalies, it would be useful if these studies could be repeated to compare an undilated fundus examination with fundus biomicroscopy to a DFE. It is possible that the better field of view and stereoscopic image provided by fundus biomicroscopy would limit the advantage of a DFE for the posterior pole in a patient with a reasonable pupil size. The second argument in favour of a DFE is that significant anomalies would otherwise be missed in the peripheral retina. However, many of the anomalies found in the peripheral retina are benign and do not need treatment (Siegel et al. 1990, Parisi et al. 1996, Batchelder et al. 1997). The question is how often does a routine DFE detect a peripheral lesion that requires treatment beyond those detected by DFEs prompted by symptoms, signs and/or risk factors. Although Siegel et al. (1990) and Parisi et al. (1996) found important peripheral lesions in patients who were asymptomatic, it is unclear whether they considered clinical risk factors that would have prompted clinicians to perform a DFE. The majority of patients with peripheral retinal disease reported by Batchelder et al. (1997) had important risk factors including previous anterior segment surgery, previous retinal detachment, strong family history of retinal detachment and high myopia. In summary, Siegel et al. (1990) suggest that a routine DFE should be part of a primary care examination. Parisi et al. (1996) support this viewpoint, particularly for paediatric examinations. Batchelder et al. (1997) suggested that pupillary dilation is only required when signs, symptoms or risk factors suggest a peripheral lesion. Finally, another aspect to consider is the legal one. It is possible that in the US, the case of Keir vs. United States will make a DFE the standard of care for paediatric examinations, particularly for the initial visit (Classé 1989).

1.3 BIBLIOGRAPHY AND

FURTHER READING

Greenhalgh, T. (2006) How to read a paper: the basics of evidence-based medicine, 3rd edn. Oxford: Blackwell.