linked to movement and change in our peripheral vision lead to rapid redirection of gaze, in order to perceive detail in this novel stimulus [48]. Depth information is acquired monocularly by relative movement and size of objects in foreground and background [32, 47], while cooperative imaging by the two eyes, with disparities as small as a few arcsec (i.e., a small fraction of the width of a foveal cone), provides detailed depth and three dimensional shape information for nearby objects [63].

•Continuous information updating through rapid involuntary eye movements (microsaccades) is built into the human visual system, which can retain an accurate image only through such frequent “refreshment:” The perception of an image stabilized on the retina would fade after a few seconds [4, 27]. Man-made image acquisition systems may not spontaneously lose image information over time, but they require a recording medium if information is to be retained.

Probably the most remarkable design accomplishment in the visual system is the fovea, or yellow spot, in the center of the retina: It combines the quality of the eye’s optics close to the main axis, the density of photoreceptor cells in the central retina, the outward displacement of secondary neural elements and blood vessels from the retinal center [1, 68], and the ingenuity of retinal and cortical connectivity and processing to achieve the highest possible image resolution within the restrictions of limited anatomic resources and physiologic bandwidth set by a biological system.

One of the greatest challenges in designing a visual prosthesis is, therefore, to reproduce the principal properties lost by eye disease or abnormal development, while using the capabilities provided by the remaining visual system to the greatest possible advantage. In order to gain a perspective of what visual prosthetics can and cannot accomplish for their recipients, it is important to understand how normal visual function depends on the anatomy and physiology of visual system component: the eye’s optics, the retina, the pathways leading from retina to visual cortex, and the cortical areas involved in visual perception. A limited understanding of the operation and role of eye movements and binocular vision is also required. Some of these topics are briefly covered here; more detail can be found in Chap. 2.

1.2  An Overview of Human Visual System Architecture

1.2.1  Architecture and Basic Function of the Eye

The structure and function of the eye correspond to the properties of visible light: Its optics form an image of the outside world on a photosensitive layer of cells, and the spectral properties of these cells construct multiple representations in different color bands. Specifically, the cornea, iris and crystalline lens (see Fig. 1.1), aided if necessary by corrective optics such as spectacles or contact lenses, cooperate to form a focused image on the back wall of the eye. The retina, a thin film covering the posterior half of the interior eye wall, contains multiple cell populations capturing

Fig.  1.1  Cross section through the human eye, showing the principal structures referred to in the text. Reprinted from [19], with permission

the image, pre-processing it for efficient information compression, and encoding it for transmission via the optic nerve to the visual centers in the brain.

Figure 1.1 shows a horizontal cross-section of the adult human eye, including the optical elements, the retina, and the optic nerve. Light entering the eye is refracted by the cornea and the crystalline lens, and aperture-limited by the iris. The lens is suspended in a ring of thin fibers (zonula). The ciliary muscle behind the iris through the connecting zonula, can adjust the refractive power of the lens to maintain a sharp image when objects are brought closer to the eye; a membrane called the capsular bag surrounds the lens and separates the anterior and posterior chambers. Light passes unhindered through the watery fluid (aqueous humor) in the anterior chamber between cornea and lens, through the gel-like fluid (vitreous body) in the posterior chamber, and through the inner retinal layers, to reach the photoreceptors – the light-sensitive cells that convert it into electrical and chemical signals, initiating the process of vision.

An intricate system of four straight and two oblique extraocular muscles allows the eye to be rapidly directed towards a visual target without requiring a head movement. More importantly, these muscles also allow for vergence (directing the two eyes to a common point at varying distances) and limited cyclo-rotation to counteract small rotations of the head or the scene and maintain a stable view of the world.

The photoreceptors are situated in the deepest retinal layer and they, along with other cells in the outer retinal layers, receive nutrients and oxygen via a network of small capillaries under the retina, the choroid plexus. The inner retinal layers have

their own blood supply, which is fed through blood vessels in the optic nerve head; the arteries and veins of the inner retinal blood supply form two semi-circular patterns (the lower of these so-called arcades can be seen in Fig. 5.3, Chap. 5) around the central retina (or macula), which therefore has only narrow capillaries to limit interference with light projected onto the photoreceptors. The center-most portion of the macula (the fovea) does not contain any blood vessels and is called the foveal avascular zone. No inner retinal blood supply is required in this area, due to the outward displacement of all inner retinal cells, away from the center [1, 68]. Note the slight indentation of the retina at this so-called foveal pit, where the ability to capture details in the image is greatest. Also note the cupping of the retinal surface at the optic nerve (the optic nerve head; also called physiological blind spot, as it contains no photoreceptors), allowing for optic nerve fibers – actually the axons of retinal ganglion cells – to converge and enter the supporting structure of the optic nerve.

To gain surgical access to the interior of the eye, one can make an incision in or near the cornea to enter the anterior segment, or cut the sclera (white outer wall of the eye) through the so-called pars plana, i.e., posterior to the attachment of the lens capsule, but anterior to the ora serrata, the forward edge of the retina. Retinal surgeons routinely use this latter route of access, and both the inner and outer retinal layers can be reached this way, albeit that reaching the outer retina requires an incision through the full thickness of the retina and the creation of an artificial detachment of the retina from the underlying retinal pigment epithelium (RPE) layer. In a healthy eye, re-attachment occurs naturally by resorption of subretinal fluid through the RPE. Recently, surgeons have also gained access to the outer retina by entering through the sclera behind the equator [71].

The retina forms a layered structure against the back wall of the eye, with photoreceptors (rods and cones) capturing the light; bipolar and ganglion cells passing the visual signal on towards the optic nerve, and horizontal and amacrine cells providing lateral interactions among cells in neighboring locations. Chapter 2 provides greater detail regarding the different cell types in each retinal layer and their functions; here we will limit ourselves to the major structures that allow visual function to occur.

In the normal retina, a highly structured arrangement of cells is seen in each layer. Under the retina, a layer of retinal pigment epithelium (RPE) cells fulfills the roles necessary to sustain the metabolism of the photoreceptors: The metabolic level of the photoreceptor outer segments is among the highest in the human body [36]. RPE cells supply nutrients and oxygen, regenerate phototransduction products, and digest debris shed by the photoreceptors [10].

Photoreceptors, the cells capturing the light, come in two main classes: rods, whose high internal gain allows vision at very low light levels [67], and cones, in short, medium, and long wavelength-sensitive types to allow color perception [25]. In both classes of cells the actual light capture and conversion takes place in the outer segment – indicated for the foveal cones in Fig. 1.2 by the abbreviation “COS” while the cell’s inner segment, situated in the outer nuclear layer (ONL), provides the transduction to secondary neurons and regulates cell function.

Fig. 1.2  Cross section through the human fovea, showing the dense packing of elongated cone outer segments and the absence of the inner retinal layers across the “foveal pit.” COS cone outer segments; ONL outer nuclear laye; HL Henle fiber layer; INL inner nuclear layer. The Henle fibers connect foveal cones with the outwardly displaced bipolar cells in the INL. Reprinted from [57], with permission

The distribution and packing of rods and cones varies dramatically across the retina. In the foveola, only medium and long wavelength-sensitive cones are found. In the surrounding foveal area, where the width of individual cones increases, and their packing density decreases accordingly, short wavelength-sensitive cones are also found, while rods are found only beyond the fovea. Figure 1.3, created 75 years ago on the basis of anatomical studies of donor retinas, still provides a fair representation of the density distribution of rods and cones along a horizontal line through the retina of a left eye. One may note that rod densities are highest around 20° eccentricity. Cones are distributed throughout the entire retina, in roughly constant density beyond the central macula. Due to the decreasing convergence from photoreceptors to bipolar and ganglion cells in the inner retina, the visual acuity of both day and night vision gradually diminishes towards the periphery.

1.2.2  Layout of the Retino-Cortical Pathway

The connection between the eye and the central nervous system is formed by the fibers of the optic nerve. As noted above, these fibers, whose diameter is on the order of 1 mm, are the axons of retinal ganglion cells. Inside the eye, the fibers run along the inner retinal surface towards the optic nerve head in a characteristic pattern, such that fibers of the upper and lower retinal halves remain separated, and fibers close to the horizontal meridian, but far from the nerve head, arc away from this line to allow room for fibers originating closer to the nerve head. This orderly

Fig. 1.3  Horizontal cross section through the human retina, showing the rod and cone packing densities in the normal human retina. Note the very narrow area of high cone density, the highest rod density near 10° eccentricity, and the absence of photoreceptors in the physiological blind spot. Originally in [45]; this version from [39], with permission

arrangement causes the fibers from the foveal area (which form 15 to 20% of all nerve fibers) to be located in the temporal quadrant of the optic nerve, at least for the anterior portion of its trajectory [27].

Once the axons enter the optic nerve, each fiber is encapsulated by a myelin sheath, formed by a class of cells called astrocytes; this sheath decreases the membrane conductance of the axons, increasing the conduction velocity and the length over which impulses can be conducted without severe attenuation [51]. Only at the so-called Ranvier nodes is the myelin sheath interrupted, allowing the impulses to be reinforced by virtue of the ion-gating properties of the local membrane.

A cross-section through the human visual pathways can be seen in Fig. 1.4. One may note that the predominant pathway leads from the eye to the lateral geniculate nucleus (LGN) of the thalamus, and from there to the occipital part of the cortex, while smaller numbers of fibers branch off to a tectal area, the superior colliculus, and to a number of pre-tectal nuclei. We will briefly discuss these subcortical pathways below.

Note also that the LGN and cortical areas exist in duplicate in the two halves of the brain. Each deals with one half of the visual world: The optic nerves from the two eyes meet in a structure called the optic chiasm, where fibers from the two nasal retinas cross over to combine with those from the temporal retina of the fellow eye; consequently each LGN and cortical hemisphere receive visual information from two corresponding retinal halves on their own side, and thus from the contralateral half of the visual field.

Fig. 1.4  Structure and location of the human primary visual pathways, in relation to other major brain structures. The left cerebral hemisphere, with the exception of the occipital cortex, has been removed; the left LGN is hidden by the optic radiations (arrow). Reprinted from [19], with permission

The LGN has a layered structure, with pairs of layers receiving axons of different ganglion cell types, and each layer in a pair receiving signals from one eye. Interactions between layers in the form of overall suppression when the retina in the fellow eye is stimulated layers have been demonstrated [53], but localized interactions across layers do not occur; this indicates that binocular processing required for stereopsis does not take place until the level of the visual cortex. The gateway function of the LGN, which in other mammals such as the cat appears to play a crucial role in adaptation and attention, and through which signals from the two eyes can mutually inhibit each other [29], is thought to be less prominent in primates, including humans. Yet anatomical feedback connections from a number of subcortical nuclei onto the LGN are as extensive in monkey as in cat [8], and gating functions related to circadian rhythms and other systemic conditions are therefore plausible in primates as well.

Forward pathways from the LGN lead to the primary visual cortex (V1, also called striate cortex; these fibers form the optic radiation), but also to higher visual cortical areas and to subcortical areas such as the superior colliculus (SC). The role of the extrastriate cortical pathways is still a topic of speculation and investigation;

from clinical cases it is evident, however, that patients with lesions to the striate cortex acquired after childhood retain little or no useful vision [11]. The roles of the tectal pathways, including mutual connections between cortical areas, the SC and the pulvinar, are also subject to active research. It has been found, for example, that cortical connections with midbrain areas are essential for maintaining and shifting attention, rather than for processing detailed visual information [35].

The visual cortex occupies the occipital and parts of the parietal and temporal lobes of the cerebral cortex. Like the entire cortex, it forms a highly folded structure, with a thickness of approximately 1.5 mm. It is surrounded by the cerebrospinal fluid, several layers of meninges – pia mater, arachnoid, and dura –, and the skull. Especially the skull forms an important barrier to any attempt at functional electrical stimulation of cortical cells.

Like the retina, the visual cortex is a layered structure, in which different cell groups perform different tasks. Along its two-dimensional surface, one finds an orderly mapped representation of the outside world. Contrary to the retina, however, the cortex consists of multiple areas, hierarchically organized, each of which performs a partial processing task in the analysis of the scene around us. At the present time, over 30 visual cortical areas per hemisphere are recognized in monkey, and a similar number of distinct areas is thought to exist in humans [61].

The first cortical representation, in the striate cortex, is shown schematically in Fig. 1.5. It presents a straightforward map of the visual world, but contains four major transformations:

The projection from the LGN (and thus retinal ganglion cells) onto V1 input cells has approximately constant density, which means that the central visual field is highly over-represented in the visual cortex: Roughly 20% of V1 represents the retinal fovea, and thus the central 1–2° of the visual field, with rapid drop-off of the density towards the periphery. This inhomogeneous map is conveniently expressed by the cortical magnification factor, M(¢), i.e., the number of mm of cortex devoted to 1° of retina, as a function of eccentricity [20, 34].

The folding of the human cortex, prompted by the evolutionary expansion of higher (cognitive) processing, has resulted in an arrangement where most of the peripheral visual field is represented in portions of V1 that are buried in the medial walls and sulci of the cerebral hemispheres. Only the foveal representation, situated along the border of V1 and the adjacent area V2, is exposed at the surface of the occipital cortex, approximately 1 cm above the inion, a protruding portion near the bottom of the skull bone on the back of the head. More peripheral visual field areas are represented along the medial walls of the cerebral hemispheres, and in deep sulci embedded within these areas.

The complex representation of the visual field onto area V1, combined with the lack of accessibility due to cortical folding, greatly reduces our ability to investigate and stimulate the peripheral visual field. Area V2 and several higher areas form the exposed portion of the occipito-parietal cortex, and would seem to provide better opportunities for peripheral field stimulation. However, while these areas may appear to be more easily accessible than V1, they have an equally dense pattern of sulci, which greatly vary between individuals; moreover, receptive field properties and visual field maps become increasingly complex in higher cortical areas [41].