The authors set their threshold of recognition at 60% accuracy. They found that for common objects, subjects passed this threshold between grid resolutions of 16 × 16 and 24 × 24 dots, and for scenes, subjects reached this threshold around a grid resolution of 48 × 48 dots. When using a slightly lower grid resolution, the authors observed that simple binary image processing was more helpful than use of edge detection. At higher resolutions, however, edge detection seemed to be more beneficial.

Recognition tasks, in general, appear to be sensitive to the number dots contained within a grid [21, 33, 34]. Particularly for faces, high image quality of near100% contrast, at least four gray levels, and at least 256 dots appears to be required for 80% or greater recognition accuracy [14]. This is not surprising, since the coarse traits of faces resemble each other, and successful discrimination is based on finer traits and shading.

16.5  Visually Guided Behavior

16.5.1  Hand–Eye Coordination

Along with their early visual acuity and facial recognition tasks. Dagnelie and colleagues­ conducted experiments of hand–eye coordination using both virtual reality and live video input [14]. In their virtual reality experiment, four subjects viewed a room with a table and chairs through pixelized vision with less than 20% contrast. Their task was to pick up objects off the floor and place them on the table, releasing them only when they rested on the surface. Out of 12 total attempts to transfer objects in this virtual scene, subjects were successful six times. In the live video experiment, with about 90% contrast and a 250 ms delay, subjects were able to transfer objects with the assistance of tactile feedback. Technological limitation prevented systematic studies, but these experiments did serve as an example for how prosthetic vision could be used for coordination tasks.

The same group later expanded their study of hand–eye coordination with simulated prosthetic vision, reported in Hayes et al., 2003 [21]. In this study, subjects were asked to perform two tasks. The first asked them to pour ten pieces of candy from one cup to another, without touching the second. Some subjects were able to do this successfully in the hardest condition, using a grid of only 4 × 4 dots. Only one subject required a grid of 16 × 16 dots. The authors concluded that, on average, a 6 × 10 grid of dots would be sufficient for a simple hand–eye coordination task.

The second task asked of these subjects was to cut along the outside of a black square outline on a white piece of paper. Times to completion and errors both fell with increasing grid size, where satisfactory performance was only achieved with a 16 × 16 grid. Hayes et al. reasoned that this task, unlike the first, requires constant reevaluation to acquire the position of the scissors relative to the border. The authors thus concluded that for more complex tasks the 6 × 10 grid would be insufficient, and larger and/or denser grids would be required for acceptable performance.

Dagnelie et al. and Srivastava et al. continued their studies with modified checkerboards, respectively in 2006 [15] and 2009 [29], by asking subjects to cover these targets with black checkers. Once a subject had correctly covered a target, the target would no longer be visible through the simulation grid. Dagnelie et al. found that, using a 6 × 10 grid, subjects could successfully learn how to perform this task with as little as 0.85% error, even in gaze-locked conditions.

Srivastava et al. utilized gaze-locking throughout their experiment (see Chap. 18 and [29]), and varied dot counts between 325 and 650 dots. Similar to the results of the counting task, practice seemed to substantially reduce any effects of dropout on time or error. After practice, these subjects were able to complete the task without any errors through the simulated cortical prosthesis.

As suggested by the Hayes et al. study [21], hand–eye coordination tasks seem to benefit from increases in grid size. It is unclear, however, whether this benefit is derived from a greater dot count, the increased visual span of the grid, or both. As seen with Dagnelie et al. [15] and Srivastava et al., gaze-locking and dropout in simulations do seem to mandate practice if normal performance is desired, but do not strongly hinder performance after the initial learning period.

16.5.2  Wayfinding

Cha et al. [4] provided subjects with simulated prosthetic vision similar to that in their visual acuity and reading experiments. The authors varied dot density, dot count, overall grid size (up to 1.7° × 1.7°), and the visual angle captured by the camera and projected onto this simulated prosthesis. They asked subjects to navigate through a maze with white walls, floor, and ceiling and black obstacles. The capture angle was found to be critical for this task: performance increased with this angle so long as individual stimuli did not become too small; performance declined once the capture angle was expanded past 18 times the angular subtense of the grid. At the optimal viewing angle, performance correlated well with the number of dots, almost regardless of dot density. The authors concluded that a cortical prosthesis with 25 × 25 or 32 × 32 electrodes, perceptually spanning 1.7° and incorporating 30° of a camera’s view, could be used effectively for high-contrast obstacle avoidance and wayfinding in a familiar environment.

Dagnelie et al. published a report in 2007 of a similar pair of experiments on wayfinding [13]. In the first experiment, subjects used 4 × 4, 6 × 10, and 16 × 16 grids, respectively spanning 11° × 11°, 16° × 27°, and 27° × 27°. The camera’s viewing angle was fixed at 37°. The authors found that, with increasing dot count, the subjects’ wayfinding performance improved. For experienced subjects, the 6 × 10 grid was sufficient for this task. In their second experiment, subjects used the 6 × 10 grid to navigate through a virtual environment and were presented additional parameters of dynamic noise and dot dropout. The authors observed that noise did not have a significant effect, and dropout of 30% led to a slight decrease in performance. These findings match well with those of Cha et al. [4], particularly as the only differences between the 4 × 4 and 6 × 10 grids were size and dot count, and not density.