UTILITY OF A DYNAMIC TERMINATION CRITERION IN BAYESIAN ADAPTIVE THRESHOLD PROCEDURES

ANDREW J. ANDERSON

Discoveries in Sight, Devers Eye Institute, Portland OR, USA

Abstract

Bayesian adaptive threshold procedures may be run for a fixed number of presentations, or may be stopped when the calculated confidence interval for the threshold reaches a selected limit (a dynamic termination criterion). Previous Monte-Carlo simulations have shown that a dynamic termination criterion failed to equate error distributions between reliable and unreliable observers, and showed no advantage, on average, over a fixed trial procedure of the same average number of trials. In this study, both human experimental and Monte-Carlo simulation data are presented which demonstrate that the width of the confidence interval used in a dynamic termination criteria is poorly predictive of individual thresholds that are in error.

Introduction

Measuring visual thresholds is fundamental to behavioral investigations of the visual system, and a variety of techniques exist to do this. Adaptive threshold measures, wherein the stimulus intensity chosen for a particular trial depends upon the subject’s responses to previous trials,1,2 are typically the most efficient methods for estimating thresholds. Bayesian estimators are a particular class of adaptive threshold methods that use bayesian techniques to combine prior knowledge about the expected distribution of thresholds (the initial probability density function (p.d.f.)3) with the knowledge obtained from each response made by the subject. This knowledge is stored in the form of a posterior p.d.f. The zippy estimation by sequential testing (ZEST) method is a bayesian adaptive technique that has been shown to be an accurate and efficient method for estimating visual thresholds.4 A bayesian technique similar to ZEST, called SITA,5 has already found use in clinical perimetry, albeit in a sub-optimal implementation using a fixed step size in stimulus intensity.

Bayesian techniques are normally ended in one of two ways. The simplest method is to run the procedure for a fixed number of presentations.4,6,7 Alternatively, a dynamic termination procedure may be used,2,8,9 in which the procedure is stopped when the calculated confidence interval for the threshold estimate falls below a criterion

Address for correspondence: Andrew J. Anderson, PhD, Discoveries in Sight, Devers Eye Institute, 1225 NE Second Avenue, Portland OR 97232, USA. Email: ajanderson@hotmail.com

Perimetry Update 2002/2003, pp. 191–197

Proceedings of the XVth International Perimetric Society Meeting, Stratford-upon-Avon, England, June 26–29, 2002

edited by David B. Henson and Michael Wall

© 2004 Kugler Publications, The Hague, The Netherlands

192 A.J. Anderson

level. With the ZEST technique, this confidence interval may be obtained by integrating the posterior p.d.f.2,4 The use of a dynamic termination criterion results in variability in the number of trials required for a procedure to terminate.

Previous simulation work has suggested that the usefulness of dynamic termination criteria is questionable.10 In particular, a dynamic termination criterion failed to equate the distribution of threshold errors between subjects of high and low threshold variability, indicating that the use of a constant criterion for confidence interval width does not guarantee that thresholds will be estimated with identical levels of precision. Also, the use of a dynamic termination criterion showed no advantage over a fixed trial procedure of the same average number of trials.

However, this previous work analyzing error distributions provides information only on how procedures compare on average, and so provides little insight into the relationship between threshold error and the width of the confidence interval in an individual test procedure. It is unclear whether the confidence interval width provides information about the error in the threshold estimate at a given point in time (i.e., is an estimator of instantaneous error), and therefore can be used to indicate those threshold estimates requiring further refinement by increased test presentations. For this to be true, there should be a relationship between the error in the threshold and the confidence interval at a given number of presentations. As the confidence interval is a measure of average variability, whereas the error relates to an individual threshold determination, it may be expected that any relationship between the two would be poor. It has been suggested, however, that the length of a dynamically terminated procedure depends in part upon the slope of an observer’s psychometric function,6 thereby implying that there should be a relationship between the width of the confidence interval and the observer’s psychometric function slope.

In this paper, I examine how well the confidence interval in a Bayesian estimator predicts instantaneous errors in the threshold estimate, after a fixed number of trials, for both simulation results and empirical data. In addition to further investigating the utility of a dynamic termination criterion, this analytical technique allows comparisons between the form of the simulation and empirical data to be made more readily than with the histogram methods outlined previously,10 which requires the use of large numbers of psychophysical observations and/or observers. In this way, the ‘real-world’ validity of the simulation data may be assessed; such assessment is important, as it has been shown previously that not all simulation differences translated into performance differences in the ‘real-world’.6,11,12

General methods

ZEST and simulation procedure

Details of the simulation procedure are described elsewhere.10 In brief, the ZEST procedure was performed as outlined in King-Smith et al.,4 with the slope of the psychometric function βz set to 3.5, the false negative probability δz to 0.01, and the false positive probability γz to 0.03 or 0.5 for yes/no and two-alternative forced choice (2-AFC) procedures, respectively. The subscript ‘z’ distinguishes ZEST parameters from those of the simulated observer (signified by a subscript ‘o’). The threshold

criterion ez was zero, giving a 0.64 probability for detection at threshold for the yes/ no procedure and an 0.81 probability in the 2-AFC procedure.4 Confidence intervals for the mean were determined by integrating the posterior p.d.f.2,4

An observer’s response to a stimulus was simulated with a Weibull frequency-of- seeing (psychometric) function, described in detail elsewhere.10 For each simulated run, the observer’s threshold location αo was randomized over a 30-dB (3 log unit) range and the observer’s psychometric function slope βo was randomized between 1 and 4. One thousand values of αo and βo were simulated for each psychophysical technique.

Experimental analysis

Seven experienced observers with corrected acuity of 6/6 or better and no history of eye disease were investigated, with all but one (the author) being naïve to the purpose of the experiment. A yes/no detection task of 14 presentations per stimulus was performed for 13 interleaved stimuli, whose initial estimates of threshold were evenly spaced over ±1 log units of the approximate location of threshold. Thirteen falsepositive catch trials were also included, with no subject registering any false responses. Stimuli were 10° square patches of 0.25 c/° sinusoidal gratings, oriented vertically, counterphase flickered at 25 Hz and presented in a 1000 msec raised cosine envelope. The center of the stimuli were at 15° nasal eccentricity, and there was 500 msec between stimuli. Five of the seven subjects performed the yes/no task.

In addition, a 2-AFC procedure of 40 presentations per stimulus was performed for 13 interleaved stimuli, whose initial estimates of threshold were evenly spaced over ±1 log units of the approximate location of threshold. Stimuli were similar to those described above, except that gratings were oriented at either 45° or 135° and subjects were forced to chose the orientation of each grating. This technique has been shown previously to isolate the same mechanisms as a simple detection task.13 Stimuli were presented centrally in four subjects, and at 15° nasal eccentricity in one subject.

The best estimate of each subject’s ‘true’ threshold was assumed to be the average of the threshold estimates from all 14 presentation runs in the yes/no procedure, and all 40 presentation runs in the 2-AFC procedure. All stimuli were produced on a calibrated video monitor, the details of which are described elsewhere.14 The study complied with the tenets of the Declaration of Helsinki and was approved by the institutional human experimentation committee at Legacy Health Systems, with all subjects giving informed consent before participation.

Analysis

Pearson’s product moment correlation coefficients (R2) were determined, with all coefficients for the simulation data being significant (p ≤ 0.05). Spearman rank correlation coefficients (rs) were used to non-parametrically assess the monotonicity of relationships, with a value of 1 indicating a perfect monotone relationship amongst ranks.15

Fig. 1. Relationship between the confidence interval and absolute threshold error (observer sensitivity αo minus threshold estimate) (left-hand panel) or observer psychometric function slope (βo) (right-hand panel) for a simulated eight-trial yes/no procedure. Straight lines and correlation coefficients (R2) are for linear regressions of the data. Spearman rank correlation coefficients (rs) for the data were 0.10 and 0.17 for the left and right panels, respectively. In the left-hand panel, 90% of errors are less than 2.2 dB (median = 0.8 dB) and 90% of CI widths less than 5.6 dB (median = 3.6 dB). For clarity, only half the data have been plotted.

Results

Yes/no data

As mentioned previously, it may be desirable to run an adaptive threshold procedure for more presentations when the confidence interval suggests that an estimate is variable, thereby minimizing error in the threshold. For this to be advantageous, however, there should be a relationship between the error in the threshold and the confidence interval at a given number of presentations. The left-hand panel in Figure 1 shows the confidence interval as a function of the absolute error in threshold for a simulated eight-presentation yes/no procedure. There is only a weak correlation between these two variables (R2 = 0.09), suggesting that the width of the confidence interval is a poor predictor of threshold error. The right-hand panel in Figure 1 replots the confidence interval as a function of subject variability, as given by the slope of the simulated observer’s psychometric function. Again, correlation is poor (R2 = 0.02), indicating that the width of the confidence interval is largely unrelated to an observer’s overall variability.

Figure 2 shows experimental data collected on five observers, plotted in a similar fashion to the left-hand panel of the simulation data in Figure 1. The data give the threshold error and confidence interval after eight presentations in each 14-presenta- tion run. The form of the data is similar to the simulation data presented in Figure 2 (left panel), and similarly shows a poor correlation between the error in threshold and the confidence interval in all observers. All correlations failed to reach statistical significance. That the correlations are slightly poorer than in the simulation may reflect the absence of very large threshold errors in the empirical data, as correlations decrease as the range of measurements decreases.16

Fig. 2. Experimental data from five observers, for a yes/no procedure. Errors and confidence interval widths were determined after eight stimulus presentations, with the true threshold for each subject defined as the average of all 14 presentation runs performed by the subject (see text for details). From top to bottom, data sets have been vertically displaced by +3, +1.5, 0, –1.5 and –3 dB, for clarity.

2-AFC data

Data for a 30-presentation 2-AFC procedure are analyzed in the same way as the yes/no data, above. The left-hand panel in Figure 3 shows the confidence interval as a function of the absolute error in threshold for the simulated data. Similar to the yes/no data, there is only a weak correlation between these two variables (R2 = 0.15). The right-hand panel in Figure 3 shows that correlation between the slope of the simulated observer’s psychometric function and the confidence interval is poor (R2 = 0.01).

Empirical data from five observers are plotted in Figure 4, with the data showing the threshold error and confidence interval after 30 presentations in each 40-presentation run. Correlation coefficients show a greater variation between subjects than seen in the yes/no empirical data (Fig. 2), although all subjects failed to show a strong association between error in threshold estimate and the width of the confidence interval. The range of correlation coefficients in the empirical data encompasses the correlation coefficient from the simulation data (R2 = 0.10, Fig. 3 left panel).

Discussion

The empirical data from the yes/no procedure (Fig. 2) confirm the form of the data from the Monte-Carlo simulation study (Fig. 1), and show that the confidence interval is poorly predictive of those thresholds that are in error. The empirical data from the 2-AFC procedure are more variable (Fig. 4), but are again consistent with the simulation data (Fig. 3). Therefore, this study adds further support to the findings of previous work questioning the utility of dynamic termination criteria.10

That there is only a poor correlation between the confidence interval (a measure of average variability) and the error within a single threshold determination, is not surprising on statistical ground, as mentioned previously. However, previous work has implied that the confidence interval is related to a subject’s overall variability, as

Fig. 3. Relationship between the confidence interval and absolute threshold error (observer sensitivity αo minus threshold estimate) (left-hand panel) or observer psychometric function slope (βo) (right-hand panel) for a simulated 30-trial 2-AFC procedure. Straight lines and correlation coefficients (R2) are for linear regressions of the data. Spearman rank correlation coefficients (rs) for the data were 0.23 and 0.13 for the leftand right-hand panels, respectively. In the left-hand panel, 90% of errors are less than 2.0 dB (median = 0.7dB) and 90% of CI widths less than 6.2 dB (median = 2.9 dB). For clarity, only half the data have been plotted.

Fig. 4. Experimental data from five observers, for a 2-AFC procedure. Errors and confidence interval widths were determined after 30 stimulus presentations, with the true threshold for each subject defined as the average of all 40 presentation runs performed by the subject (see text for details). From top to bottom, data sets have been vertically displaced by +10, +5, 0, –5 and –10 dB, for clarity.

measured by the slope of the observer’s psychometric function,6 rather than to a measure of instantaneous error. However, the results in this paper show that this relationship is trivial (Figs. 1 and 3, right panels), which agrees with the results of earlier simulations.10

In summary, I do not recommend that dynamic termination criteria be used in bayesian adaptive threshold methods. In addition to the limitations described in previous simulations,10 this paper demonstrates that dynamic termination criteria are poorly predictive of those thresholds requiring more information to be gathered, i.e., those thresholds estimates that are in error.

References

1.Watson AB, Fitzhugh A: The method of constant stimuli is inefficient. Perception Psychophys 47:8791, 1990

2.Treutwein B: Adaptive psychophysical procedures. Vision Res 35:2503-2522, 1995

3.Spahr J: Optimization of the presentation pattern in automated static perimetry. Vision Res15:12751281, 1975

4.King-Smith PE, Grigsby SS, Vingrys AJ, Benes SC, Supowit A: Efficient and unbiased modifications of the QUEST threshold method: theory, simulations, experimental evaluation and practical implementation. Vision Res 34:885-912, 1994

5.Bengtsson B, Olsson J, Heijl A, Rootzén H: A new generation of algorithms for computerized threshold perimetry, SITA. Acta Ophthalmol Scand 75:368-375, 1997

6.Madigan R, Williams D: Maximum-likelihood psychometric procedures in two-alternative forcedchoice: evaluation and recommendations. Perception Psychophys 42:240-249, 1987

7.Kontsevich LL, Tyler CW: Bayesian adaptive estimation of psychometric slope and threshold. Vision Res 39:2729-2737, 1999

8.Watson AB, Pelli DG: QUEST: a Bayesian adaptive psychometric method. Perception Psychophys 33:113-120, 1983

9.Harvey LO Jr: Efficient estimation of sensory thresholds. Behav Res Methods Instruments Computers 18:623-632, 1986

10.Anderson AJ: Utility of a dynamic termination criterion in the ZEST adaptive threshold method. Vision Res 43:165-170, 2003

11.Shelton BR, Picardi MC, Green DM: Comparison of three adaptive psychophysical procedures. J Acoust Soc Am 71:1527-1533, 1982

12.Simpson WA: The step method: a new adaptive psychophysical procedure. Perception Psychophys 45:572-576, 1989

13.Anderson AJ, Johnson CA: Mechanisms isolated by frequency-doubling technology perimetry. Invest Ophthalmol Vis Sci 43:398-401, 2002

14.Anderson AJ, Johnson CA: Effect of dichoptic adaptation on frequency doubling perimetry. Optometry Vis Sci 79:88-92, 2002.

15.Hays WL: Statistics. New York, NY: Holt, Rinehart and Winston Inc 1963

16.Bland JM, Altman DG: Statistical methods for assessing agreement between two methods of clinical assessment. Lancet 8:307-310, 1986

PULSAR PERIMETRY: A NEW PROCEDURE FOR EARLY GLAUCOMA EVALUATION

Preliminary findings

ANA M. FERNÁNDEZ-VIDAL, JULIÁN GARCÍA-FEIJOÓ and

JULIÁN GARCÍA-SÁNCHEZ

Hospital Clínico San Carlos, Madrid, Spain

Abstract

Purpose: To evaluate preliminary results with Pulsar perimetry in patients with ocular hypertension. Methods: Pulsar perimetry is a new perimetric procedure, which uses stimuli combining spatial resolution (SR) and contrast (C) for the evaluation of early glaucoma. The stimulus properties are aimed at isolating large retinal ganglion cells. Results: For normal individuals, mean sensitivity (MS) for Pulsar perimetry was 21.23 src (spatial resolution and contrast units) with an SD of 2.55. Mean defect (MD) was 1.1 src SD 1.56 and loss variance (LV) was 5.87 src SD 2.83. For patients with ocular hypertension: MS was 18.27 src SD 2.99; MD was 2.98 src SD 2.55 and LV was 9.9 src SD 6.59. MD and LV differences between the two groups were statistically significant (p < 0.05) with 95% confidence limits of (-2.81; -0.95) and (-6.26; -1.80), respectively. The area under ROC curve obtained was of 0.76. Conclusion: Pulsar perimetry may have greater sensitivity for the detection of early defects in patients with ocular hypertension than conventional perimetry.

Introduction

Pulsar perimetry is a new perimetric procedure, which has been developed by Dr Gonzalez de la Rosa’s group. It uses either moving or pulsing stimuli combining spatial resolution (SR) and contrast (C) for the early evaluation of glaucoma.1 The prototype has the ability to examine various visual functions (spatial resolution, contrast perception, motion, and temporal modulation), which are aimed at isolating large retinal ganglion cells.2-10 The reason for this choice of stimulus is that the magnocellular system is thought to be affected early in glaucoma.11-14

Pulsar perimeter uses white round stimuli, 5º in diameter, 500 msec long, shaped as a wave decreasing in amplitude (Fig. 1), with a mean luminance equal to that of the background (100 asb).

The prototype consists of a high resolution examination screen made up of a photometrically calibrated 21-inch computer monitor with a resolution of 1600 x 1200

Address for correspondence: Ana M. Fernández-Vidal, MD, Siguero #59, Madrid – 28035, Spain. Email: anafvidal@hotmail.com

Perimetry Update 2002/2003, pp. 199–205

Proceedings of the XVth International Perimetric Society Meeting, Stratford-upon-Avon, England, June 26–29, 2002

edited by David B. Henson and Michael Wall

© 2004 Kugler Publications, The Hague, The Netherlands

V = Cont. x cos ((2π x SR x D)- π) x (1- (D/R))

R (grados)

D (abs. grados)

Cont. (dB)

V

SR (ciclos/grado)

Fig. 1. Stimulus used in Pulsar perimetry.

pixels, vertical frequency of 60 Hz and color temperature of 6500ºK. This allows the examination of 66 points in the central visual field: 30° in the nasal and temporal regions and 24° in the superior and inferior areas. The stimulus wave can be modulated in spatial resolution, contrast, color, centrifugal motion velocity or frequency of oscillation as previously described.1

For this study, a temporal modulation program with pulsing stimuli at 30 Hz (phase- counter-phase oscillations) was used (program T30W). A scale of 36 logarithmic spatial resolution-contrast units (src) was used. The stimuli increase in difficulty in both spatial resolution and contrast at the same time (Fig. 2). The range varies from 0.5 cpd of spatial resolution and 100% contrast (level 0 src) to 6.3 cpd spatial resolution and 6% contrast (level 35 src). The dynamic range is adequate for the different spatial resolution abilities of the different retinal areas,1 and stimulus presentation rate is adapted to the patient’s response time, with random pauses to discourage rhythmic responses.

The initial results in normal subjects have been published.1 Both inter-individual variability and short-term fluctuation, as well as threshold increases with age, have similar characteristics to conventional perimetry.1 Sensitivity decreases more rapidly from the center to the periphery (Fig. 3), but still has enough dynamic range to be able to discriminate between normal and pathological results.1

The purpose of this study was to evaluate our first results with Pulsar perimetry in patients with ocular hypertension, and compare them to normal individuals.

Methods

For this study we used a temporal modulation perimetry (program T30W), which used white pulsing stimuli with a frequency of oscillation in phase and counter-phase of 30 Hz, which theoretically stimulate the magnocellular pathway. However, because the program uses spatial frequencies which are adapted to each visual field area, it probably also stimulates the parvocellular system. Although the perimeter is designed

Fig. 2. Scale used in Pulsar perimetry.

Fig. 3. Example of a pulsar visual field in a normal individual: sensitivity decreases faster from the center to the periphery.

to use a bracketing strategy, we chose to use the TOP strategy, since our patients are used to short examination times.

Forty-one left eyes of 41 normal patients were selected from our glaucoma service. We chose patients with experience in performing visual fields who had normal conventional white-white visual fields, and divided them into two groups: those with an intraocular pressure of 21 mmHg or lower, and those with more than 21 mmHg. They were recruited according to the following inclusion criteria:

• age > 18 years

•best corrected VA > 0.7 for each eye

•previous perimetric experience

•optic disc within normal limits

•patients with any of the following were excluded from our study: o significant lens opacities

o refractive defect >3 D spherical or >1.5 D astigmatic o pupil size < 3 mm

o ocular pathologies or surgery

o diabetes or neurological diseases

o secondary causes of ocular hypertension (OHT) o absence of perimetric reliability

All the patients selected for this study went through a complete ophthalmological examination, which included conventional automated white-white perimetry with an Octopus 1-2-3 perimeter, TOP strategy, and G1 program (Interzeag AG, Switzerland). Approximately one week later, Pulsar perimetry was performed after a short demonstration program to familiarize each patient with the new stimulus.

For the statistical analysis, we used the SPSS program (10.1 version) and means were compared using Student’s t test for independent data.

Results

The mean age, sensitivity, MD and LV can be found in Table 1. MD and LV differences between the two groups were statistically significant (p < 0.05) with 95% confidence limits of (-2.81;-0.95) and (-6.26; -1.80), respectively. The area under ROC curve obtained was 0.76 (Fig. 4).

Table 1. Pulsar perimetry: MS, MD, and LV obtained from normal individuals and patients with OHT

N: number; src: spatial resolution and contrast units

Discussion

Pulsar perimetry has been designed to examine functions that are thought by some to be affected early in glaucoma. As a first step to evaluating the usefulness of this technique, we have designed a simple study to compare the mean values of the three most frequently used parameters in perimetry: MS, MD and LV.

both studies using higher risk patients, such as those with asymmetric glaucoma, and longitudinal studies would be very helpful for determining whether these patients with normal conventional perimetry and pathological Pulsar perimetry will develop clinical glaucoma in the future.

Our next goal is to carry out further studies to compare the results of this study with those obtained using other types of perimetry also designed for the early evaluation of glaucoma, such as short wavelength blue-on-yellow perimetry,19-22 flicker perimetry,23-27 and frequency-doubling technology perimetry.28-30

References

1.González-Hernández M, Pareja Ríos A, Rodríguez M, González de la Rosa M: Combined spatial resolution and contrast perimetry in normal subjects. In: Wall M, Mills R (eds) Perimetry Update 2000/2001, pp 109-114. The Hague: Kugler Publications 2001

2.Sample PA, Madrid ME, Weinreb RN: Evidence of a variety of functional defects in glaucoma suspects. J Glaucoma 3(Suppl 1):5-18, 1994

3.Livingstone M, Hubel D: Segregation of form, color, movement and depth: anatomy, physiology and perception. Science 240:740-749, 1988

4.MacLeod DI, Williams DR, Makous W: A visual nonlinearity fed by single cones. Vision Res 32:347363, 1992

5.Levi DM, Klein SA: Sampling in spatial vision. Nature 320:360-362, 1986

6.Virsu V, Rovamo J, Laurineh P, Näsänen R: Temporal contrast sensitivity and cortical magnification. Vision Res 22:1211-1217, 1982

7.Hart WM Jr, Silverman SE, Trick GL, Nesher R, Gordon MO: Glaucomatous visual field damage. Luminance and color-contrast sensitivities. Invest Ophthalmol Vis Sci 31:359-367, 1990

8.Sample PA, Weinreb RN: Color perimetry for assessment of primary open-angle glaucoma. Invest Ophthalmol Vis Sci 31:1869-1875, 1990

9.Motolko MA, Phelps CD: Contrasts sensitivity in asymetric glaucoma. Int Ophthalmol 7:45-50, 1984

10.Quigley HA, Sanchez RM, Dunkelberger GR, Green WR: Chronic glaucoma selectively damage large optic nerve fibers. Invest Ophthalmol Vis Sci 23:913-920, 1987

11.Johnson CA: The Glenn A. Fry Award Lecture: early losses of visual field function in glaucoma. Optom Vis Sci 72:359-370, 1995

12.Turano KA, Huang AS, Quigley HA: Temporal filter of the motion sensor in glaucoma. Vision Res 37:2315-2324, 1997

13.Quigley HA, Dunkelberger GR, Green WR: Chronic human glaucoma causing selectively greater loss of large optic nerve fibers. Ophthalmology 95:357-363, 1988

14.Quigley HA, Green WR: The histology of human glaucoma cupping and optic nerve damage: clinicopathologic correlation in 21 eyes. Ophthalmology 86:1803-1827, 1979

15.González de la Rosa M, Martinez A, Sanchez M, Mesa C, Cordovés L, Losada MJ: Accuracy of tendency oriented perimetry (TOP) with the Octopus 1-2-3 perimeter. In: Wall M, Heijl A (eds) Perimetry Update 1996/1997, pp 119-123. Amsterdam: Kugler Publications 1997

16.González de la Rosa M, Martínez Piñero A, González Hernández M: Reproducibility of the TOP algorithm results versus those obtained with the bracketing procedure. In: Wall M, Wild JM (eds) Perimetry Update 1998/1999, pp 51-58. The Hague: Kugler Publications 1999

17.Martínez A, Pareja A, Mantolan C et al: Results of the tendency oriented perimetry in a normal population. Vision Res 36:153-156, 1996

18.González de la Rosa M, Morales J, Weijland A: A Comparison of the Tendency Oriented Perimetry Method with Normal Threshold Perimetry Using a PC Controlled Octopus 1-2-3 Perimeter. Schlieren, Switzerland: Interzeag AG 1998.

19.Johnson CA, Adams AJ, Lewis RA: Automated perimetry of short-wavelength mechanisms in glaucoma and ocular hypertension: preliminary findings. In: Heijl A (ed) Perimetry Update 1988/1989. Proceedings of the Eighth International Perimetric Society Meeting, pp 31-37. Amsterdam: Kugler

&Ghedini 1989

20.Casson EJ, Johnson CA, Shapiro LR: Longitudinal comparison of temporal modulation perimetry

with white-on-white and blue-on-yellow perimetry in ocular hypertension and early glaucoma. J Opt Soc Am (A) 10:1792-1806, 1993

21.Johnson CA, Adams AJ, Casson EJ: Blue-on-yellow perimetry: a five-year overview. In: Mills RP (ed) Perimetry Update 1992/1993, pp 459-465. Amsterdam: Kugler Publications 1993

22.Johnson CA, Adams AJ, Casson EJ, Brandt JD: Progression of early glaucomatous visual field loss as detected by blue-on-yellow and standard white-on-white automated perimetry. Arch Ophthalmol 111:651-656, 1993

23.Tyler CW: Specific deficits of flicker sensitivity in glaucoma and ocular hypertension. Invest Ophthalmol Vis Sci 100:135-146, 1981

24.Austin MW, O’Brien CJ, Wishart PK: Flicker perimetry using a luminance threshold strategy at frequencies from 5-25 Hz in glaucoma, ocular hypertension and normal controls. Curr Eye Res 13:717723, 1994

25.Matsumoto C, Okuyama S, Otsuky T, Uyama K, Otori T: Automated flicker perimetry in glaucoma. In: Mills RP, Wall M (eds) Perimetry Update 1994/1995, pp 141-146. Amsterdam: Kugler Publications 1995

26.Lachenmayr BJ, Tothbacher H, Gleissner M: Automated flicker perimetry versus quantitave static perimetry in early glaucoma. In: Heijl A (ed) Perimetry Update 1988/1989. Proceedings of the Eighth International Perimetric Society Meeting, Vancouver, 1988, pp 359-386. Amsterdam: Kugler & Ghedini 1989

27.Yoshiyama KK, Johnson CA: Which method of flicker perimetry is most effective for detection of glaucomatous visual field loss? Invest Ophthalmol Vis Sci 38:2270-2277, 1997

28.Sponsel WE, Arango S, Trigo Y, Mensah J: Clinical classification of glaucomatous visual field loss by frequency doubling perimetry. Am J Ophthalmol 125:830-836, 1998

29.Iester M, Mermoud A, Schnyder C: Frequency doubling technique in patients with ocular hypertension and glaucoma: Correlation with octopus perimeter indices. Ophthalmology 107:288-294, 2000

30.Johnson CA: The frequency doubling illusion as a screening procedure fo detection of glaucomatous field loss. ARVO abstract. Invest Ophthalmol Vis Sci (Suppl) 36:S335, 1995