18.6  Psychophysical Studies with the Estimated Maps

In the field of visual prosthetics, very limited psychophysical studies have been conducted to judge the performance of a cortical prosthesis. After the studies by Cha and colleagues, many psychophysical studies were conducted targeting other areas of vision restoration or for judging different image processing schemes, but detailed psychophysical studies addressing cortical structure and biological limitations have been missing [4–6]. In 2007 psychophysical tests were conducted at John Hopkins University by Srivastava and colleagues for testing the expected response of a cortical prosthesis, being developed at the Illinois Institute of Technology, Chicago, using 650 intracortical electrodes targeting the dorso-lateral surface [20]. Tests were conducted on five volunteer subjects, three men and two women, using a dotted phosphene map similar to Fig. 18.3. Individual variations in the cortex and failure of some electrodes to elicit a phosphene were also incorporated in these studies.

Area and layout of the visual cortex may vary by up to 50% between individuals [7], hence the anatomical area for electrode implantation will similarly vary, hence in few patients the area available for electrodes might be less than the average. In addition, some electrodes, or stimulation sites, might fail during the surgical procedure itself, or during long-term implantation. To study the effect of fewer phosphenes than might be expected from the electrode count, dropout effects were included in our studies. Performance was judged under 0% dropout, 25% dropout and 50% dropout conditions. Three different tasks were selected to observe if, with these limited-field phosphene maps, persons can adapt to perform different tasks of eye-hand coordination and mobility. These tasks were judged to be representative of the basic tasks required to be done in daily day-to-day life. These experiments were conducted using eye-tracking to simulate the normal movement of phosphenes in visual space.

The purpose of the first two psychophysical experiments were to determine the subject’s ability to perform detection by counting the white fields on a checkerboard and eye hand coordination by placing black checkers on the white fields of the checkerboard. The accuracy of counting and placing along with the time taken to complete the task were used to judge performance. Subjects were able to inspect the board by scanning the head-mounted camera in parallel with the board, or change their viewing angle by tilting their heads. The decision by the subjects for scanning the boards was intuitive for each individual. By giving the subject the ability to scan the board, spatial and temporal integration was achieved. The experimental setup is shown in Fig. 18.5. The leftmost image shows the phosphene map for the dorso-lateral surface with 650 electrodes implanted on a region of 3 cm radius area, with occipital pole as the center. The center image shows the checker board used, and the rightmost image shows the dotted image seen by the test subjects in a headset with eye tracking.

Counting time for counting all the white fields and the number of fields reported by the subject were recorded for each trial. The significant factors effecting the study were individual differences in subjects (F = 25.29, p < 0.0001), increase

Fig. 18.5  Checkered board as observed through a phosphene-like map

in practice (F = 23.58, p < 0.001), dropout (F = 8.41, p = 0.0040) and increase in complexity (F = 50.75, p < 0.001). The dropout had a weak significance compared to the other factors showing that the effects of dropout could be overcome by practice. The same analysis was done for each individual subject, and practice was the significant factor in all subjects except the low vision subject.

Similar to counting time, placing time was also recorded. The significant factors effecting placing timing were variation due to individuals (F = 14.02, p < 0.0001), increase in practice (F = 30.12, p < 0.001), the dropout (F = 9.03, p = 0.0040) and the number of white fields on the board (F = 109.42, p < 0.001). Dropout had a weak significance compared to the other factors. For individual analysis, all the subjects show a significant reduction in task time with increasing practice. Three subjects showed an increase in the time with an increase in the dropout level, but with a modest significance level. This shows that the practice effect dominates over the dropout effect, and a decline in performance due to dropout can be overcome by practice.

In the third experiment, subjects’ ability to recognize a pathway and orient oneself to follow it, without memorizing or recall, was judged by observing the performance of the subjects’ maneuvering in virtual mazes. While for the counting and placing tasks, the subjects were observing the checkerboards with a camera, and could therefore adjust the level of detail by changing their viewing distance and angle, the full scene of the virtual mobility experiment, was imaged onto the phosphene map, and subjects were trained to use a game controller to change their vantage point, as if moving through virtual space. Thus, depending on the task, the acceptance angle of the camera or other image source may have to be adjusted by the subject to get an optimum view. This can also be achieved by selecting the part of the captured image to be presented in the phosphene map. This is shown in Fig. 18.6 where the virtual maze is shown on the left, which was made to fit on the central phosphene map by adjusting the image scale in the graphics processor. The resulting dotted image is shown in the right image.

The effect of learning on the performance was observed with increased practice of the experimental task. The time to complete each maze and the number of wayfinding errors were recorded. The factors affecting the experiment were individual differences (F = 34.21, p < 0001), increase in practice (F = 90.04, p < 0001), and the dropout (F = 6.06, p = 0.0189). Dropout had a low significance, similar to the counting

Fig. 18.6  Phosphene-like image of virtual maze

and placing experiments, and a separate analysis checking the effects within individuals yielded practice as the only significant effect. Other factors and interactions had no statistical significance.

The results from these experiments demonstrated that even with a limited number of phosphenes in one visual hemifield, with limited eccentricity in visual space, it is possible to attain a level of proficiency with which prosthesis wearers can perform simple tasks, albeit with practice.

A significant finding from these studies was that the degradation in the image, due to a lower number of phosphenes (higher dropout), could be largely overcome by an increase in practice and use by the tested subject. This result can be significant for individuals with a small area of visual cortex available for electrode implantation, who may still be candidates for prosthesis implantation. This result also is reassuring for researchers who worry about the failure of electrodes, or neuronal stimulation sites, during surgical implantation and over the lifetime of a prosthesis, and what effect this may have on performance. These results also may be of value to visual prosthesis researchers using other substrates, such as the retina, optic nerve, or lateral geniculate nucleus (LGN). As an indication for what they can expect from such devices. They may also provide a guide to vision scientists conducting simulation studies, in regard to how biological response and surgical difficulties might affect their results.

As human studies are conducted and more real maps are obtained, similar experiments might be repeated using more realistic maps, and potentially providing better predictions of performance. The experiments conducted by Cha and colleagues gave 625 electrodes for getting a limited sense of vision. In the experiments mentioned in this chapter by Srivastava and colleagues, it was observed that even with 325 electrodes, and an incorporated estimate of cortical distortion in the phosphene maps, the subjects learned and adapted to performing the basic tasks [20].

Another observation of this study was that producing a large number of phosphene in a limited visual field might not be helpful, since many of them might potentially overlap, and the effective result would not be a major improvement in recognition ability. It would be more helpful to create distinct phosphenes on larger visual area, which might require development of improved surgical techniques.

The relatively crude and limited information provided by the cortical visual prosthesis under development by various research groups, if successful, might help blind subjects obtain some assistance for conducting simple tasks in daily life.

References

1.Brindley GS (1970), Sensations produced by electrical stimulation of the occipital poles of the cerebral hemispheres, and their use in constructing visual prostheses. Ann R Coll Surg Engl,

47(2): p. 106–8.

2.Brindley GS (1982), Effects of electrical stimulation of the visual cortex. Hum Neurobiol,

1(4): p. 281–3.

3.Brindley GS, Lewin WS (1968), The sensations produced by electrical stimulation of the visual cortex. J Physiol, 196(2): p. 479–93.

4.Cha K, Horch K, Normann RA (1992), Simulation of a phosphene-based visual field: visual acuity in a pixelized vision system. Ann Biomed Eng, 20(4): p. 439–49.

5.Cha K, Horch KW, Normann RA (1992), Mobility performance with a pixelized vision system.

Vision Res, 32(7): p. 1367–72.

6.Cha K, Horch KW, Normann RA, Boman DK (1992), Reading speed with a pixelized vision system. J Opt Soc Am A, 9(5): p. 673–7.

7.DeYoe EA, Carman GJ, Bandettini P, et al. (1996), Mapping striate and extrastriate visual areas in human cerebral cortex. Proc Natl Acad Sci USA, 93(6): p. 2382–6.

8.Dobelle WH, Mladejovsky MG (1974), Phosphenes produced by electrical stimulation of human occipital cortex, and their application to the development of a prosthesis for the blind.

J Physiol, 243(2): p. 553–76.

9.Dobelle WH, Mladejovsky MG, Girvin JP (1974), Artificial vision for the blind: electrical stimulation of visual cortex offers hope for a functional prosthesis. Science, 183(123): p. 440–4.

10. Dobelle WH, Turkel J, Henderson DC, Evans JR (1979), Mapping the representation of the visual field by electrical stimulation of human visual cortex. Am J Ophthalmol, 88(4): p. 727–35.

11. Dougherty RF, Koch VM, Brewer AA, et al. (2003), Visual field representations and locations of visual areas V1/2/3 in human visual cortex. J Vis, 3(10): p. 586–98.

12. Holmes G (1918), Disturbances of vision by cerebral lesions. Br J Ophthalmol, 2(7): p. 353–84. 13. Horton JC, Hoyt WF (1991), The representation of the visual field in human striate cortex.

A revision of the classic Holmes map. Arch Ophthalmol, 109(6): p. 816–24.

14. Kaido T, Hoshida T, Taoka T, Sakaki T (2004), Retinotopy with coordinates of lateral occipital cortex in humans. J Neurosurg, 101(1): p. 114–8.

15. Lee HW, Hong SB, Seo DW, et al. (2000), Mapping of functional organization in human visual cortex: electrical cortical stimulation. Neurology, 54(4): p. 849–54.

16. Levy I, Hasson U, Avidan G, et al. (2001), Center-periphery organization of human object areas. Nat Neurosci, 4(5): p. 533–9.

17. McFadzean R, Brosnahan D, Hadley D, Mutlukan E (1994), Representation of the visual field in the occipital striate cortex. Br J Ophthalmol, 78(3): p. 185–90.

18. McFadzean RM, Hadley DM, Condon BC (2002), The representation of the visual field in the occipital striate cortex. Neuroophthalmology, 27(1–3): p. 55–78.

19. Schmidt EM, Bak MJ, Hambrecht FT, et al. (1996), Feasibility of a visual prosthesis for the blind based on intracortical microstimulation of the visual cortex. Brain, 119 (Pt 2): p. 507–22.

20. Srivastava NR, Troyk PR, Dagnelie G (2009), Detection, eye-hand coordination and virtual mobility performance in simulated vision for a cortical visual prosthesis device. J Neural Eng,

6(3): p. 035008.

21. Stensaas SS, Eddington DK, Dobelle WH (1974), The topography and variability of the primary visual cortex in man. J Neurosurg, 40(6): p. 747–55.

22. Wandell BA, Brewer AA, Dougherty RF (2005), Visual field map clusters in human cortex. Philos Trans R Soc Lond B Biol Sci, 360(1456): p. 693–707.