Ординатура / Офтальмология / Английские материалы / Artificial Sight Basic Research, Biomedical Engineering, and Clinical Advances_Humayun, Weiland, Chader_2007

4

Prosthetic Vision Simulation in Fully and Partially Sighted Individuals

Matthias Walter1, Liancheng Yang2 and Gislin Dagnelie2

1Biomedical Optics and Ultrafast Lasers, Kirchhoff Institute für Physik, Ruprecht Karls Universität

2Department of Ophthalmology, Johns Hopkins University School of Medicine

Abstract: This article examines performance and learning under conditions of simulated prosthetic vision, using the tasks of counting white squares on modified checker boards, and placing black checkers on these squares. Aim of our study was to determine how much environmental information people would be able to obtain from crude pictures without tactile feedback. For this, we used video camera images which were convolved by a dedicated computer program in real time to represent phosphene vision as future chip implants might be able to do. The resolution was equivalent to a vision of 20/2400. The program allowed us to vary a broad set of variables such as contrast, number of gray levels, number of phosphenes per column and row, etc. In our tests we used a matrix of 6×10 dots with Gaussian intensity profile. Test subjects varied in age, gender, and educational background.

Introduction

Human vision is mediated by one of the most highly developed sensory systems found in nature. The capacity to combine high spatial resolution near the center of fixation with a wide peripheral field of view, accurate depth perception, color discrimination, and light–dark adaptation over 12 orders of magnitude is unparalleled [1]. The information from over 100 million photoreceptor cells is pre-processed in subsequent layers of retinal neurons, so that 1 million fibres in the optic nerve are sufficient to transport the information to the visual cortex, where further information processing takes place.

The most common causes of adult visual impairment in North America and Western Europe include age-related macula degeneration (AMD), retinitis pigmentosa (RP), and diabetic retinopathy. AMD is caused by a combination of genetics and environmental and other individual factors, and is characterized by progressive loss of central vision. Six million Americans are affected, and it is

71

72 Walter et al.

the leading cause of blindness among individuals over age 50. About 15 million more Americans are pre-symptomatic. For RP there is currently no effective treatment or cure. Early symptoms of this retinal degeneration include night blindness and increasing loss of peripheral vision. These symptoms most often begin in the first three decades of life and are followed by a progressive loss of vision over several decades, often leading to complete blindness. [2]. The disorder is inherited and about 100,000 people in the US are affected. Worldwide one in every 4000 persons is afflicted with RP.

Blindness caused by these disorders is due to failure of the outer retina. i.e. the photoreceptors and retinal pigment epithelium (RPE) layers. Secondary nerve cell layers throughout the retina appear to remain viable for a period of years or even decades [3]. Recent studies by Marc et al. show that deprivation of input from the photoreceptor layer leads to rewiring in the nerve cell layer [4]. This plasticity results in new connections of the secondary neurons which may extend up to several hundred micrometers, corresponding to approximately 1 in visual space. This has an important implication for prosthetic vision: Either the rewiring of the secondary neurons has to be stopped as early as possible or the wearer of a visual prosthesis may at best see only a phosphene pattern with size and spacing on the order of 1 .

A prosthesis could replace the function of the degenerated photoreceptors by stimulating the remaining secondary retinal neurons, the optic nerve fibers, or the visual cortex. Over the past few years, several groups have shown that it is possible to stimulate the inner retina with small electrodes, producing a sensation of points of light called phosphenes. The location of the stimulated area on the retina and the visual field location where the elicited phosphene is perceived match closely [5, 6].

In the experiments reported here, we assumed that the wearer of a retinal prosthesis will be able to see a regular field of fuzzy dots. With this assumption in mind, we modeled prosthetic vision by convolving camera images with a very low-resolution spatial grid filter, in real time, and presenting the results to sighted volunteers. The experiments included tasks in which vision had to guide motor activities (hand–eye coordination). Subjects’ performance under these conditions of extremely poor vision was recorded.

Methods

The experiments were performed during an internship of Matthias Walter at the Lions Vision Center, Department of Ophthalmology, Johns Hopkins University, Baltimore, MD, USA.

Equipment

The headset in this study is a modified Low Vision Enhancement System (LVES; no longer in production) head-mounted display (HMD). It has a 36 ×48 field of

view (8 mm exit pupil) and is capable of monochrome VGA resolution (640×480 pixels). The headset has built-in refractive correction, IR pupil illumination, and a charge coupled device (CCD) camera allowing recording of the test subject’s eye movements. The scene in front of the HMD wearer was captured by a CCD camera, Watec model WAT-660D (Genwac, Orangeburg, NY), mounted centrally on the front cover of the headset, as shown in Figure 4.1. The optical axis of the camera is at a 90 downward angle to the subject’s line of sight. The test subject was allowed to use just the left eye (the right eye monitor of the headset was switched off), hence there was an offset of about 3.2 cm between the axis of the camera view and the subject’s eye position, the camera position did correspond to the subject’s cyclopean eye and body axis. The video feed from the downward-looking camera was processed by a Dell Inspiron 8200 laptop computer with a dedicated filter implemented under the DirectShow operating environment.

Stimulus

The DirectShow filter program allowed many different settings such as size of the phosphene grid and of each phosphene, gray levels of phosphenes and background, etc. In our experiments we used a grid of 6 × 10 dots with 1 effective diameter and 2 center-to-center spacing, representative of prosthesis resolutions that may be achieved in the near future. The dots had a Gaussian intensity profile on a black background. The dot raster was shown to subjects only while they were allowed to perform the experimental task.

Figure 4.1. Angle between the view planes of the camera and the test subject and the offset between the middle of the camera view and the middle of the test subjects view.

74 Walter et al.

Modified checkerboards (8 × 8 squares) were printed on white paper with a square size of 2×2 cm. Only a small number of squares (1 to 16) were white; the rest were black. The white squares were positioned in quasi-random positions limited only by the fact that they were not allowed to share a common border. The boards were framed by a 2 cm white border to define the area of interest for the test subjects. The circular checker pieces used in the experiments were made of black plastic and had a diameter of 2.2 cm. This size was chosen so that coverage of a white square was accomplished without too much interference from checkers in diagonally adjacent positions.

Experimental Procedures

Test subjects participated in the experiments on a weekly basis, in one hour-long test sessions. The one-hour limit was chosen to limit errors caused by fatigue, while the weekly schedule was kept to maintain a sufficient level of practice. We noticed in the first few test sessions that subjects had more problems to get back to their former performance level if this follow-up interval was stretched beyond a weekly schedule.

The test subjects were told only the following facts about the experiment:

•The checker boards consist of 8 × 8 squares; most of them black, some white; there is also a white rim around the whole board to define the area of the 8 × 8 squares.

•Your first task is to count the number of white squares; later you will be given a number of black checkers that either matches or exceeds the number of white squares on the board and you will have to cover the white squares with these for at least 50%. If the coverage is less than 50%, the square will be counted as not covered.

•We will measure the time you take to complete the task, but the main focus is for you to correctly count the number of white squares, and to achieve sufficient coverage of the white squares by the checkers.

•The white squares may be placed directly adjacent to the white rim or even in the corners of the board; two or more white squares may be placed diagonally so that they share one corner, but they will never share a border.

•The size of the squares perceived in the headset depends on the distance between the camera and the board; you will be looking straight ahead, but the camera is looking down at the board on the table.

•Tilting your head may help you in finding squares that are incompletely covered but it will also distort the image you see.

•Your hand will show as dark in the pixelized image; this means that it will obstruct your view if it is between the camera and the board, which may lead you to believe that you covered a square with a checker while you actually did not do so.

During the first few sessions, these facts were repeated every time before testing was started. This was done not only to remind subjects of the test conditions, but

also to assure them that there had been no changes compared to the previous time they did the experiment. During the entire series of test visits, the subjects never saw the boards other than through the headset, in filtered form, which forced them to create their own mental image. They also were never given feedback regarding their performance, to simulate the condition of total self-learning faced by a blind prosthesis wearer, and to see what problem-solving strategies each subject would develop. To decrease the probability of pattern recognition for the counting task, we designed three different arrangements of white squares for the boards with 4 or fewer white squares. Board designs can be found in the Appendix. To further decrease possible pattern recognition, we randomly rotated the boards by 180 . Checkerboards were named according to their orientation and the number of white squares. Thus “Top 3A” refers to a board with 3 white squares in standard orientation; while “Bottom 3B” would refer to a different board with 3 white squares, turned 180 .

Time to completion of the counting and checker placing tasks was measured with a stopwatch. Only during this time was the video feed from the camera switched on, so subjects never had an opportunity to observe the removal, placement, or rotation of the board. The board was always removed between trials even to just be turned around, so no sound cues were given to the subject as to what action was taken. Figure 4.2 shows a pixelized view of a checker board.

Tests were split into three parts:

•Experiment 1: Subjects familiarized themselves with the test equipment, the boards, and checkers and the two different tasks of counting white squares and placing the checkers.

•Experiment 2: Subjects practiced their skills in counting white squares while they were given the boards in random order.

Figure 4.2. Picture of the view the test subject has. It shows the upper left edge and parts of the rim (seen from the position of the test subject) of the board and some white squares.

76 Walter et al.

•Experiment 3: Subjects practiced their skills in placing checkers while they were given the boards in random order.

Subjects

We are reporting results from five test subjects (3 male, 2 female; aged 25– 51 years) who were all practiced in the use of simulated phosphene vision through their participation in other tests (reading tests and face/object recognition) that did not involve object manipulation. Four subjects were normally sighted; the remaining subject was severely visually impaired (legally blind) due to a chemical burn of both corneas in early childhood.

All testing was done according to the tenets of the Helsinki convention. The subjects were informed of the purpose and procedures of the experiment, and read and signed a consent form approved by the IRB of the Johns Hopkins University School of Medicine. In accordance with HIPAA rules the identities of the test subjects were coded with numbers to mask their identities.

Results

To evaluate the results of the tests the following rules were applied:

1. Errors in counting tasks were “punished” by using the formula

FinalTime =UsedTime+

Real number of squares – Counted number of squares

where UsedTime is the time measured with the stop watch, Real number of squares is the number of white squares on the board, Counted number of squares is the number that was called out by the test subject, and AverageTime is the calculated time it takes the test subject to count one white square on a board. This AverageTime is calculated for every test subject respectively using the formula:

with Sum of time as the sum of the measured time for all the boards in the session and Sum of squares as the total number of squares summed over all boards used in the session.

2.To evaluate the “placing checker” parts of the experiments the following approach was chosen:

Errors were divided into groups with their respective error values:

78 Walter et al.

Figure 4.3. These graphs show the results of the counting task for all the test subjects and the linear regression of the data (solid straight line). Board Top 16 was used only once at this stage of the experiment, so there is no standard deviation data available and thus no error bar. Top 1 was one time used once without noting the time to familiarize the test subjects with the task. The number of counts for the other boards varied from test subject to test subject but was at least 3 times for every board and at the maximum 6 times (small numbers at the x-axis denote if there were more than three times and give the number of times). The dashed line shows the regression of the calculated average time value (see Eq. 2).