draw their percepts on a tilted drawing board 57 cm in front of them, while fixing their gaze on the center of the board. In a later phase of the study the subject was seated 50–200 cm from a white wall and was asked to indicate with a laser pointer to the location of the phosphene relative to a reference point on the wall. Subjects also traced the outline of the phosphenes with the laser pointer. The experimenter redrew the phosphene with a pencil. Interestingly, while retinotopy was clearly present in the sighted subjects and the subject with partial vision loss, the retinally blind subject did not show clear retinotopy and had a degraded spatial representation. In contrast, Brindley and Lewin [3] found a clear retinotopy in their blinded patient when using subdural electrodes. The authors speculate that the diffuse cortical excitation inherent in TMS makes precise stimulation impossible when cortical organization is interrupted due to total vision loss [5].

In a similar TMS study the authors made use of a digitizing tablet connected to a personal computer to directly convert the drawings to digital data [14]. After each TMS pulse the subject drew the image on the tablet, which was provided with a central pin for tactile reference to the center of the visual field. This phosphene mapping method showed that TMS was capable of evoking phosphenes in 17 out of 18 sighted people, and that phosphenes could be evoked along the entire visual field by stimulating the occipital cortex with single pulses. Blinded subjects often needed TMS pulse trains instead of single pulses, and in only 54% of these subjects phosphenes could be evoked. TMS is generally used to disrupt cortical function and although many phosphenes appeared as dots of light, spots of darkness (“scotomas”) were also reported.

19.6  Recent Simulation Studies Using Phosphene Mapping

Simulation studies on the functionality and capabilities of visual prostheses are becoming more important (Chap. 16). Regarding phosphene mapping, simulation studies can be used to carefully control the test environment and allow a comparison of different mapping strategies.

19.6.1  Tactile Simulations at Shanghai Jiao Tong University

Ren and associates published two papers about novel ways to construct absolute phosphene maps based on simulation studies. Both studies used normally-sighted subjects who were presented with simulated phosphenes using a head-mounted display (HMD). The first study [4] made use of a touch screen (39 cm in width, 30 cm in height) that was placed at eye level. The screen was provided with a tactile reference point in the center of the screen. Subjects were seated in the dark and fixated at the center of the screen by means of a chin rest. Their left index finger was placed on the reference point on the touch screen for tactile feedback, while the

right index finger was used to point at the simulated phosphene on the touch screen, much like the pointing hemisphere deployed some 40 years earlier by Brindley and Lewin [3]. The experiment compared two test conditions by presenting phosphenes with and without a reference grid projected in the HMD, which divided the visual field in 6 × 8 cells. Under both conditions, the experiment was preceded by a training phase in which the subjects could see their own response on the touch screen. In the first phase of training, phosphenes were presented in a predictable way, allowing the subjects to familiarize themselves with the equipment. In the second training phase, phosphenes were randomly presented. After that, the actual test was performed during which subjects could not see their response. Phosphenes were ­presented at 3°, 11° and 15° eccentricity [4].

The investigated parameters included dispersion of the responses (standard error of the response in mm), accuracy (the distance between phosphene and mean response in mm) and response time. Their results showed that in the presence of the reference grid dispersion, accuracy and response time tended to be lowest. In addition, dispersion and response times increased when phosphenes were presented at larger eccentricity. The authors also showed that dispersion was larger in the left two quadrants of the visual field compared to the right two quadrants. They attributed this result to the fact that the left hand was always used for tactile reference which interfered with pointing to a phosphene in the left half of the visual field.

In a follow-up study, Ren and colleagues [31] used a very similar setup, but adapted their method to improve tactile feedback to the subject by overlaying a 19″ touch screen monitor with a 31 × 31 push-button array (41 cm in width, 35 cm in height). Tactile references were improved by (a) an elevated center button representing the origin and (b) slightly elevated buttons along the horizontal, vertical and diagonals of the push-button array. Subjects could use both hands to localize the phosphene on the push-button array. In contrast to their earlier method, the screen with the push-button array was placed horizontally on a table in front of the subject. Training consisted of three phases. The first training phase permitted the subjects to familiarize themselves with the array by letting them feel the origin and the elevated buttons indicating the dividing lines. The second training phase provided the subjects with phosphenes localized in a restricted portion of the visual field. The third and final training phase provided the subject with 24 random phosphenes and the subject could observe the response in the HMD. The test phase consisted of 98 randomly generated phosphenes. Again, dispersion, accuracy and response time were recorded.

Compared with the unaided touch screen method dispersion was lower and accuracy more constant when using the push-button array. Furthermore, the systematic left hemifield error observed in the first study was absent. However, response times were much higher (25 vs. 3 s in the earlier study) and the authors speculate that subjects spent most of this additional time on finding the tactile ­references and appropriate button on the array. Another possibility is the fact that the push-button array was placed horizontally in front of the subject, instead of vertically at eye level as in their first study. This setup likely demanded more of the subjects, since they had to translate visual field coordinates to a horizontal surface.

Though testing times may become long, the push-button array may prove a valuable method for testing subjects with little or no residual vision, because of the tactile references, better resolution and reduced localization errors.

19.6.2  Simulations in Our Laboratory

Dagnelie and Vogelstein developed and compared three different phosphene mapping methods based on phosphene localization simulation studies in four normally-sighted volunteers [6, 7]. An HMD with a 40° × 50° binocular display with 5 arcmin resolution was used to present phosphenes (see Chap. 16). The HMD provided subjects with a central fixation point and was equipped with a pupil tracker to monitor eye movement. Trials were aborted if the gaze deviated by more than 0.5°. All three methods were designed to mimic a prosthesis positioned over the primary visual cortex of one cerebral hemisphere by presenting 32 randomly located phosphenes binocularly in one visual hemifield at eccentricities up to 20°. Phosphenes were round dots with a diameter of 20 arcmin at the fovea and increasing in diameter to 40 arcmin near 20° eccentricity in order to mimic cortical magnification. For each subject, one set of phosphenes was generated for use in all three tests to facilitate comparison between methods within a subject. Additional random sets were used in the touch screen and eye movement procedures, discussed below. Subjects came in for four or five 1-h sessions.

The first method was much like the method of Chai et al. discussed earlier, deploying a touch screen (18″ × 12″, height × width). Figure 19.1 shows a subject performing this procedure. Subjects sat in front of the screen and held their left index finger on a tactile marker located halfway down the left edge of the screen. Subjects were told to place their right index finger immediately beside the left index finger on the screen and to align their fixation point in the HMD with this (unseen) finger as best they could. A phosphene was then presented, accompanied by a tone and subjects had to slide their right index finger across the screen to the location of the phosphene and lift their finger off the screen. A second tone signaled that the computer had registered the lift-off coordinates. This process was repeated until all 32 phosphene positions were mapped; this process was repeated, for a total of three estimates per position. All data were obtained in a single session.

The second approach recorded a saccade to the remembered phosphene location, using a calibrated pupil tracker in the HMD. The subject fixated on the central dot displayed in the HMD. After a warning tone, a phosphene momentarily appeared (400 ms) after which a saccade was made to the former phosphene location. The subject was required to briefly maintain gaze while the pupil-tracking software recorded the coordinates of the final eye position. This procedure was repeated for all 32 positions and repeated three times in a single session.

In contrast to the absolute phosphene coordinates estimated with the first two techniques, the third method constructed a relative phosphene map. This so-called triadic distance comparison method compares distances among point triads, with map reconstruction through multidimensional scaling (MDS). During the test, subjects

Fig. 19.1  Subject performing the touch screen task employed by Dagnelie and co-workers. The left index finger is placed on a tactile marker representing the center of fixation, placed midway along the left side of the touch screen. The head-mounted display shows an internal fixation point, and the subject attempts to keep the line of sight in the HMD aligned with the tactile fixation marker. A pupil tracking camera inside the HMD is used to monitor steady central fixation while a phosphene is present. The subject’s right index finger has just completed tracing towards the perceived phosphene location and is briefly held steady before being taken off the screen, marking the position. The scene camera, visible on the front of the HMD, was not used in this test

again maintained fixation on the central dot on the HMD screen. Three phosphenes appeared sequentially and remained visible for 500 ms. Subjects numbered the phosphenes according to their appearance and reported which two dots were closest and which two were farthest apart. The experimenter keyed in the reply and started a new trial. Testing all possible triads including the 32 phosphene locations would have required 4,960 trials. To reduce this number, the 32 phosphenes were divided into four overlapping groups of 16 from which pseudo-random triads were presented. Each pair in a group of 16 was presented four times, rather than the maximum of 14 times. In this way, 160 triadic comparisons per group were made, for a total of 640 trials to complete all four groups; maps from the groups of 16 could be combined by virtue of the eight common points among “adjacent” groups of 16. Performing the 640 comparisons required two or three sessions.

The MDS procedure consisted of building a similarity matrix for all pairs, in which ternary values were assigned for each response (“2” for closest, “1” for intermediate and “0” for the farthest pair), similar to the method from [26] described in Sect. 19.3. A Kruskal MDS procedure was then performed to reconstruct the two-dimensional dot distribution (mean = 0, SD = 1) [23]. Note that the resulting map not only needs to be translated and scaled to allow comparison of the reconstructed coordinates with those obtained by the touch screen and eye movement methods but, due to the relative nature of triadic comparisons, it will also require rotation, and possibly a mirror imaging operation.

Figure 19.2 shows results of a complete set of tests for one representative subject. Touch screen results are shown in panel a, eye movement results in panel b, triadic comparison results in panel c, and combined results in panel d. The square symbols and connecting (dashed) lines in each figure represent the stimulus locations­ and the order in which they were presented; the connecting lines are presented­ only to facilitate comparison with the corresponding mean responses (drawn black lines; no data points, since each break point in the line is the center of gravity of the responses for that stimulus in multiple trials). Panels a and b show raw responses, while those in panel c have been transformed to optimally fit the stimulus coordinates, as explained above.

A comparison between the three methods suggests that the touch screen and eye movement tests had better relative accuracy with increasing eccentricity, but overall had poor reproducibility (up to 25% test-retest variability). Furthermore, as judged by comparing the line sets in each panel, the touch screen data show a relative expansion along the horizontal axis, and a downward trend, while the eye movement test resulted in a horizontal compression; the break points in the triadic comparison reconstruction map appear much closer to the phosphene coordinates, but this may in part be due to the optimized scaling. For the combination and comparison in panel d we have therefore optimized the fits of the touch screen and eye movement data through translation and isotropic expansion. Averaging all three methods in panel d yields a map in which the lines representing the “grand mean” response bears a good, albeit somewhat distorted, resemblance to the lines connecting the phosphene coordinates.

While these results are those for a single subject, the findings are typical of what was found in a half dozen others: touch screen responses overestimate horizontal eccentricity, while eye movement responses underestimate them. Triadic distance comparison test performance was better than the other two tests. This does not mean, however, that one can rely on this test alone: one or more absolute mapping methods are required to obtain an overall map of phosphene locations.

Also, given the time-consuming nature of the triadic comparison tests, it may be best to concentrate efforts using that test on clusters of closely spaced phosphenes while using the absolute techniques to establish relationships between such clusters. By applying translation, rotation and scaling to achieve maximum correspondence among data from different tests, one can hope to attain the most accurate maps.

One should bear in mind that in an actual prosthesis wearer there will be no stimulus map to which the results can be fitted. Nonetheless the results obtained in our lab inspire some confidence. Dagnelie and coworkers [7] computed a distortion metric from the distance estimation errors for all possible phosphene pairs across the three tests for all five subjects tested with a uniform phosphene set. Three ­subjects had distortion scores under 15% for all tests. Combining maps by averaging the data of all three tests reduced the errors below 10%, which should enable adequate image recognition. The authors conclude that the three procedures, especially in combination, permit the construction of distortion maps with sufficient fidelity to enable shape recognition by future prosthesis wearers.