the observer which 1/32 of the area covered by the phosphene array contains the target. In this latter case, we would expect the observer to deviate from the target by less than that amount; the mean deviation would be reduced by approximately a factor of 4 (rather than 16, since the field is two-dimensional).

17.5  Discussion

We have reviewed two studies of ours that address image analysis, its use in microelectronic retinal prostheses, and the perception of low-resolution images. These studies form the beginning of an approach that integrates theory and experiment and aims to better constrain the design of a prosthesis. Effectively, the approach seeks to answer the question, “How should high-resolution images be analyzed before rendering phosphene images?”

In our psychophysical experiments, subjects fixated and pursued a small, moving target that was rendered on an array of phosphenes [8]. There, we showed that image analysis, which converts the high-resolution image of the moving target to the phosphene image, can indeed be used to improve subjects’ performance. During trials, subjects scanned the phosphene array over the target, using some phosphenes in preference to others. We modeled this scanning using a bivariate function, and we termed this model the “artificial preferred retinal locus” (APRL). The experiments in [6] were numerical. There, we used the APRL in conjunction with various image analysis schemes and measured the information contained in the phosphene image. We found that the scheme affording subjects the best tracking performance imparted the most information to the phosphene image. Further, we found an optimal scheme of Gaussian kernels for image analysis which we predict would afford further improvements in performance.

Our approach contributes to the existing literature on image analysis, its use in microelectronic retinal prosthesis, and the perception of low-resolution images. We have established an exchange between information theory and visual modeling, that is, the simulation of prosthetic vision using normal observers. This exchange allows for the design and implementation of image analysis schemes on the basis of quantitative reasoning, and for those schemes to be verified via psychophysical methods. For example, our numerical experiment predicts that the optimal Gaussian scheme for fixation and pursuit involves kernels with standard deviation equal to 0.6 times the phosphene-to-phosphene spacing [6]. This prediction may be tested in a visual modeling experiment, prior to a test in actual implantees who are capable of performing simple visual tasks. Alternative approaches to image analysis often simply cite “biological inspiration.” For example, an edge-detection scheme is often thought to be justified because edges are known to be of particular salience to the visual system. These approaches may have merit, but the design of the image analysis scheme seems arbitrary, and usually is not tested using visual modeling.

It is important to consider the computational cost of an image analysis scheme. For our purposes, the operation of a kernel on an image involves A real multiplications

and A − 1 real additions, where A is the area of the kernel in pixels. Therefore, the relative cost of image analysis scheme Q2 is proportional to the area of the Q2 kernel divided by that of the Q1 kernel. In many image processing applications, Gaussian kernels are restricted to a circular support with a radius of three standard deviations. Our circular averaging kernels (Q1) had a diameter equal to the separation of phosphenes. Therefore, the ratio of areas of these kernels is 3.24. That is, in our tracking study [8], the computational cost of using Q2 was 3.24 times that of Q1.

So far, our approach involves models of scanning that vary in space, but not time. However, scanning is likely to be better described by models that are spatiotemporal. A spatiotemporal model of scanning would describe not only which phosphenes were used in preference to others, but how the outputs of many phosphenes were used in combination over time. In other words, a spatiotemporal scanning model would describe how subjects tended to sweep the phosphene array across the high-resolution target. Our psychophysical data suggest that the temporal nature of scanning is important (see also [3]). For example, the image analysis scheme Q0 compelled subjects to use nystagmus-like scanning, rapidly moving the array back and forth across the underlying target. Rather than using the information contained in the phosphene image at a single instant, subjects integrated the phosphene array activity over short periods, and used that integrated information to guide behavior. Developing scanning models, that is, APRLs, to include secondand higher-order statistics is the subject of ongoing work.

Our approach concerns visual fixation and pursuit. In the future, we aim to extend the approach to include other tasks, such as reading. To do so, our psychophysical experiment [8] could be modified to involve the identification of commonly used words, as opposed to the tracking of a small, moving target. In this new experiment, subjects would employ scanning behaviors that were specific to reading. Then, images of these commonly used words could be used as stimuli in the numerical set-up of [6], and an image analysis scheme could be tailored to these images. Overall, it is likely that different image analysis schemes would apply to different visual tasks. For example, a Gaussian scheme, like Q2, may be suited to tracking a small, moving target, but some other scheme involving some other class of kernels may be better suited to reading, for example, oriented Gabor functions.

17.6  Conclusion

We have discussed our approach to image analysis, microelectronic retinal prostheses, and the perception of low-resolution images. We believe that this approach can be used to help constrain the design of an implant. The approach is analogous to the acoustic modeling of cochlear implants which involves normally hearing listeners. That approach has made important contributions to the improvement of clinical outcomes in cochlear implantees since 1990. We hope that our visual modeling approach, and developments thereof, will ultimately contribute to improved clinical outcomes in retinal implantees.

Acknowledgments  We thank Shaun Cloherty for comments on an early draft of the manuscript.

References

1.Blahut RE (1987), Principles and practice of information theory. Addison-Wesley: Norwood, MA.

2.Cha K (1992), Functional capabilities with a pixelized vision system: application to visual prosthesis. PhD dissertation, University of Utah.

3.Chen SC, Hallum LE, Suaning GJ, Lovell NH (2007), A quantitative analysis of head movement behaviour during visual acuity assessment under prosthetic vision simulation. J Neural Eng 4: p. S108–S123.

4.Dowling J (2005), Artificial human vision. Expert Rev Med Devices 2: p. 73–85.

5.Fornos AP, Sommerhalder J, Rappaz B, et al. (2005), Simulation of artificial vision, III: do the spatial or temporal characteristics of stimulus pixelization really matter? Invest Ophthalmol Vis Sci 46: p. 3906–3912.

6.Hallum LE, Cloherty SL, Lovell NH (2008), Image analysis for microelectronic retinal prosthesis. IEEE Trans Biomed Eng 55: p. 344–346.

7.Hallum LE, Dagnelie G, Suaning GJ, Lovell NH (2007), Simulating auditory and visual sensorineural prostheses: a comparative review. J Neural Eng 4: p. S58–S71.

8.Hallum LE, Suaning GJ, Taubman DS, Lovell NH (2005), Simulated prosthetic visual fixation, saccade, and smooth pursuit. Vision Res 45: p. 775–788.

9.Rubinstein JT, Miller CA (1999), How do cochlear prostheses work? Curr Opin Neurobiol 9: p. 399–404.

10. Thompson, Jr., RW, Barnett GD, Humayun MS, Dagnelie G (2003), Facial recognition using simulated prosthetic pixelized vision. Invest Ophthalmol Vis Sci 44: p. 5035–5042.

11. Timberlake GT, Mainster MA, Peli E, et al. (1986), Reading with a macular scotoma. I. Retinal location of scotoma and fixation area. Invest Ophthalmol Vis Sci 27: p. 1137–1147.

Chapter 18

Simulations of Cortical Prosthetic Vision

Nishant R. Srivastava

Abstract  Cortical stimulation for restoring vision presents researchers with many challenges and questions. The extent of the human visual cortex varies up to 50% from one individual to another, cortical folding and sulci limit the area of implantation, and surgical difficulties make it difficult to implant electrodes to produce phosphenes in the whole visual space. Researchers are faced with question such as: which electrodes to use – surface electrodes that are easy to implant or intracortical fine-metal electrodes that have lower current requirements and have five times better resolution? How many phosphenes will be enough to give limited, but useful vision? How will cortical physiology affect phosphene maps? Will percepts be distinct dots or complex in nature? What will be the long term response to stimulation? Will the brain adapt to seeing through dotted images? Some of these questions can be answered by conducting human psychophysical tests.

Abbreviations

In the 1990s, Cha et al. simulated arrays varying from 100 electrodes (10 × 10 arrays) to 1,024 electrodes (32 × 32 arrays), represented by small dots in a video display mounted on ski goggles to test the requirements of a cortical prosthesis

N.R. Srivastava (*)

Department of Biomedical Engineering, Pritzker Institute of Biomedical Science and Engineering, Illinois Institute of Technology, 3255 S. Dearborn, WH 314, Chicago, IL 60616, USA e-mail: srivnis@gmail.com