Chapter 19

Phosphene Mapping Techniques for Visual Prostheses

H. Christiaan Stronks and Gislin Dagnelie

Abstract  Mapping of the visual world onto the visual system occurs in a highly ordered manner, yet with substantial interindividual variability. Since the retinal map of the scene at the photoreceptor level is fully determined by the optical projection of the eye, it is likely that a proximal map generated by a retinal prosthesis closely adheres to the same geometric projection. Once the nerve signals enter the optic nerve, this orderly map is redistributed, and while maps at more proximal levels still follow general rules, special mapping techniques in individual LGN or cortical prosthesis recipients will be required to allow reconstruction of spatial ­relationships in the outside world by means of a disorderly array of phosphenes.

This chapter provides an overview of mapping techniques that have been used in a number of laboratories; discuss the strengths and weaknesses of each; and suggest ways in which various techniques can be combined.

Abbreviations

19.1  Importance of Mapping

Ever since researchers first started eliciting phosphenes in blind patients through electrical stimulation there has been a need to specify the location of the phosphenes in the visual field. Over 30 years ago phosphene mapping was defined by

H.C. Stronks (*)

Lions Vision Research and Rehabilitation Center, Wilmer Eye Institute, Johns Hopkins University School of Medicine, 550 N. Broadway, 6th floor, Baltimore, MD 21205, USA e-mail: hstronk1@jhmi.edu

Everitt and Rushton as “determining the position of each phosphene in the visual field” [13]. Phosphene mapping is an important step in determining the functionality of visual prostheses. Mapping phosphenes allows the characterization of how evoked phosphenes by stimulation of the different electrodes of a visual prosthesis cover the visual field. After the phosphene map is obtained, clinical fitting procedures can be applied to adjust the visual input stage (e.g., the video image) and visual processing strategies to provide the prosthesis wearer with a proper percept representative of the outside world.

In general, there are two different approaches to obtain a phosphene map; absolute and relative phosphene mapping. Absolute maps describe the position of phosphenes in absolute coordinates in the field of view, while relative maps provide information about the spatial relationships between phosphenes, in terms of distance­ and angle. Both methods have advantages and disadvantages, as discussed below.

Retinal prostheses likely yield predictable phosphene maps, since the representation of the outside world on the retina (i.e., the retinotopical organization) is determined by simple geometry and is constant across subjects [12]. Nevertheless, retinal neural organization has been shown to change during prolonged periods of visual impairment (e.g. [15, 25] and Chap. 3), so phosphene mapping might still prove important in patients with long-term vision loss. Since retinotopy is largely preserved in the optic nerve, implants in the optic nerve are likewise expected to yield predictable phosphene maps, although the accuracy and stability of such a map will depend on the ability to precisely position the electrodes, due to the high density and thin caliber of the nerve fibers.

Phosphene maps obtained in cortical prosthesis users are relatively unpredictable, since multiple maps of the visual field are represented in different cortical areas which may cause widely spaced electrodes to evoke phosphenes in different cortical areas, where they may or may not fall in the same area of the visual field [11]. Moreover, the presence of sulci and gyri in the cortex may lead to unexpectedly large distances between phosphene locations in the visual field. Finally, cortical organization differs from person to person and, more importantly, the functional organization in longer-term visually impaired individuals may be substantially different from normal-sighted people due to the plasticity of the visual cortex [22]. Therefore, phosphene mapping will be especially important in cortical prosthesis recipients.

This chapter deals with various phosphene mapping techniques. Comparable mapping techniques will be discussed together and will be presented in roughly chronological order, starting with the earliest report on mapping techniques from Brindley and Lewin. Wherever possible, comparable studies (e.g., cortical and ­retinal prostheses, simulation studies etc.) will be discussed together in the text. The chapter concludes with results from our laboratory, a short overview of ­phosphene mapping methods, and suggestions will be made which methods to use in different situations.

19.2  Early Absolute and Relative Mapping Procedures

in Subjects with Cortical Prostheses: Pointing Techniques

Mapping of phosphenes already proved to be highly informative during the ­pioneering work of Brindley and Lewin. They acknowledged the importance of absolute and relative phosphene mapping in the 1960s. In one of their studies they chronically implanted a subject with no functional vision with an array of 80 subdural extra-cortical electrodes [3]. Absolute maps were obtained by letting the subject point towards the perceived phosphene with the left hand in a hemispherical bowl with a radius of 0.59 m, i.e., approximately at arm’s length. The right hand grasped a small knob inside the bowl for tactile reference. Relative maps were obtained by sequentially stimulating two electrodes and asking the subject to describe the spatial relations between the two phosphenes, such as distance and compass angle.

The early experiments of Brindley and Lewin already showed that phosphene maps were not a simple reflection of the electrode array projected into the visual field of the subject. Rather, phosphene maps roughly corresponded to the classical cortical maps constructed by examination of gunshot victims of WWI (e.g. [17]), that showed the nonlinear projection of the visual field onto the cortical surface. Their experiments also showed that these phosphene maps were not very regular. Phosphenes lying in a straight line in the visual field could be evoked by electrodes lying in a triangular configuration on the cortical surface. It was also shown that activation of distant electrodes could result in phosphenes overlapping in the visual field. In addition, stimuli delivered well above threshold by a particular electrode could result in additional phosphenes being elicited in distant locations. These early experiments strongly indicated the need for proper phosphene mapping, since phosphene configuration may differ substantially from electrode organization.

Dobelle and Mladejovsky [10] performed similar experiments in acute sessions on normally sighted patients undergoing occipital lobe surgery. One patient received a sub-chronic implant for a period of 2½ days and most of the data were obtained from this subject. Phosphene mapping was performed by letting the subject point to where the phosphene was perceived. Maps were then created by drawing the phosphenes in a visual field map. Relative maps were obtained by asking the subject to describe how different phosphenes interrelated. Though the phosphene mapping techniques are not discussed in detail, the authors provide detailed analyses of the phosphene maps they obtained and include a critical discussion on the mapping­ techniques employed.

In conjunction with the classic cortical maps, they found that for a given interelectrode distance, phosphene spacing varied depending on the area of the cortex being stimulated. Phosphenes close to the center of the field of vision (e.g., elicited by stimulation of electrodes near the occipital poles) were usually closer together than those in the periphery. Moreover, when the electrode array spanned a cortical fissure (sulcus), a gap between phosphenes in the visual field was observed, which

could be explained by the fact that the electrodes did not penetrate deep enough to stimulate the cortical tissue within the sulcus [10].

The authors also recognized some of the most important advantages and disadvantages of the phosphene mapping techniques employed both by them and by Brindley and Lewin. While absolute mapping provides the scale of the map, relative mapping provides the detailed interrelationship of phosphenes [3], making both techniques complementary. Relative mapping is more time consuming and may therefore not be suitable for acute testing in the operating room. Nevertheless, relative mapping using (near-) simultaneous phosphene presentation may be preferable over absolute maps obtained by sequential activation of different electrodes, since phosphenes move with eye-position, making absolute localization in the visual field difficult. Another general disadvantage of (absolute) mapping by pointing is that phosphenes elicited by different electrodes may be too close together to be resolved properly due to inaccuracies in pointing, especially in blind subjects who have no visual feedback [10, 26].

The pointing method described in the 60s by Brindley and Lewin was applied in much the same way by Gothe et al. [16], who investigated cortically evoked phosphenes by means of transcranial magnetic stimulation (TMS). TMS is a method to affect cortical activity, and sometimes evoke phosphenes, by electromagnetic stimulation through the intact scalp and skull. Gothe et al. instructed sighted subjects and individuals with residual vision to use a laser pointer in a room with dimmed lights to indicate the position of phosphenes onto a semicircular screen that was placed 120 cm before the subject and extending 33° on each side. Subjects without residual vision were instructed to point in the direction of the percept.

They found that the number of cortical locations from which phosphenes could be evoked increased with the amount of residual vision. In normal-sighted subjects, stimulation of all areas of the occipital lobe yielded phosphenes, while in totally blind subjects only 20% proved responsive. These results were somewhat surprising, since Brindley and Lewin reported that activation of almost all of their 80 electrodes yielded perceivable phosphenes [3].

19.3  The Computer Era: Refining the Pointing Method of Phosphene Mapping

Following the pioneering studies of Brindley and Dobelle, relative phosphene mapping was improved during the 1970s when computers became available. Handmade maps could be digitized using the relative coordinates of each phosphene [9]. Everitt and Rushton [13] proposed a method to combine all the available data in a patient by digitizing and pooling relative maps and using an iterative “best fitting” procedure to obtain a reliable relative phosphene map. They were actually able to use these maps to present figures and letters by direct electrode stimulation which yielded patterns that were recognizable by the subject.

Dobelle and associates worked out a fully computerized protocol to overcome the problems of eye drift and inaccuracy of pointing [26]. Subjects were presented with pairs of simultaneously evoked phosphenes, minimizing the effect of eye-drift. One electrode was stimulated for 1 s, the other 3 s and the subject was asked to report the spatial relationship between the “short flash of light” and the “long flash of light”. The subject entered the relative position of two phosphenes into a computer through two key presses on a touch-tone telephone pad, using “5” as a reference key. Thus, “1” encoded “above and left”, and “2” “directly above” etc. By mapping different phosphene combinations according to their relative X and Y coordinates, the authors were able to construct relative phosphene maps.

This procedure was applied on a male patient blinded by a gunshot wound who was subsequently chronically implanted with a subdural 64-electrode array on the occipital cortex (knows as the striate cortex, V1, or area 17). The computerized procedure not only resulted in a relative phosphene map, it also enabled the authors to construct an accurate and detailed layout of the cortical surface under the electrode array [11]. With this map they could accurately predict where sulci were situated under the electrode array and even how deep a given sulcus was by determining the magnitude of the shift in phosphene location of adjacent electrodes. Boundaries of the striate and peristriate cortical areas could be identified by a reversal in phosphene direction when adjacent electrodes were stimulated (these areas contain reversed maps of the visual field). The calcarine fissure along the medial wall of the occipital lobe – known to separate cortical areas representing areas above and below the horizontal meridian – could accurately be identified by a sudden shift in adjacent electrodes evoking phosphenes in the upper and lower visual field. Furthermore, electrodes that could elicit phosphenes in different visual field locations, depending on the current level, were found to lie alongside sulci: the intervening portion of the visual field projected to the portion of cortex in the sulcus, and could therefore not be activated with the surface electrodes used at the time.

The methods for phosphene mapping essentially did not change much during the decades following Dobelle’s work. Bak and colleagues [1] mapped size and absolute position of phosphenes by instructing intracortically stimulated (sighted) subjects during acute recordings to fix their gaze and point with their finger on a white screen with calibrated markers where the phosphene was perceived. Later, for absolute phosphene mapping in a chronically implanted subject, they used a dart board with 12 sectors and five annular zones for tactile feedback. The subject was asked to place a dart at the location of the phosphene while keeping her gaze fixed. For relative mapping they used the computerized method from Dobelle and associates and improved the resolution by deploying a joystick that could detect 16, instead of eight, relative angles [28]. The latter method was supplemented with verbal information to incorporate spacing between phosphenes.

The Brussels group of Veraart published several papers in which they mapped phosphenes by letting a subject point to the evoked phosphenes. Phosphenes were evoked with a four-electrode optic-nerve prosthesis that was chronically implanted in a subject suffering from retinitis pigmentosa without useful light perception [8, 29, 30]. The four electrodes were positioned around the right optic nerve.

Absolute phosphene mapping was performed using a method very similar to that described by Brindley and Lewin 30 years earlier. The chronically implanted subject was instructed to point to the location of the perceived phosphene in a hemisphere with a radius of 0.45 m. While the task was performed, the volunteer’s head was steadied in front of the hemisphere using a frame that provided support for the forehead, chin and parietal skull. The subject’s index finger was placed on the fixation point (a disc in the center of the hemispheric surface) as a proprioceptive reference. The subject was instructed to fix her gaze at the (unseen) fixation point and eye movements were recorded with a camera. Furthermore, electro-oculograms were assessed to monitor eye movements. To help the subject identify phosphenes, electric stimuli were preceded and followed by a tone. The fingers of the right hand were used to indicate the perceived phosphene as a shape on the hemisphere. Various phosphene characteristics were recorded such as position, dimensions and motion.

Interestingly, dependent on the exact stimulation parameters, 64 different phosphenes varying in shape and size could be elicited. More importantly, the phosphenes covered a visual angle of about +35° to −50° vertically and −30° to +30° horizontally, despite the fact that the implant contained just four electrodes.

Subsequent studies on this subject were performed using phosphene mapping methods very similar to the first study. The subject’s gaze was steadied and monitored in the same way and phosphenes were localized similarly using the pointing hemisphere. Mapping was performed by an observer who copied the azimuth and elevation coordinates from the hemisphere with the aid of meridians and parallels traced on the hemisphere. These data were then transferred to a digital database in which phosphenes were described as pixels with 1° resolution. Phosphene area was defined as the number of pixels within the phosphene.

Again, like in the previous study, phosphenes covered a large area of the visual field (from −30° to +30° horizontally and +20° to −50° vertically), despite the presence of only a very limited number of electrodes. Exact location depended mainly on the electrode position and current level. Each quadrant of the visual field was mostly accommodated by one electrode and higher current levels evoked phosphenes closer to the fixation point. Position was also influenced by duration, number of pulses and pulse rate of the applied pulse trains. Phosphene size and luminosity did not clearly depend on any parameter, but tended to increase at higher stimulus levels. No relation was found between stimulating condition and phosphene color or “texture” [8, 30].

In a later publication they increased the total number of individually addressable phosphenes in the phosphene map to 109, excluding the additional “ghost” phosphenes that appear at higher current [2]. By mathematically fitting model equations relating phosphene location to the afore-mentioned parameters, stimulus conditions could be calculated and assigned to an electrode to elicit a specific phosphene along the visual field. Fitting a camera and processing strategies that translated the perceived image into a phosphene map actually enabled the subject to identify simple patterns. Even though little is known about the long-term stability of phosphenes in such a crude prosthesis, these findings illustrate the importance of accurate phosphene mapping strategies: Phosphene maps proved critical in translating the subjective percepts into discrete maps that could be used for clinical fitting of the visual ­prosthesis into a functional device.