Bahram Khoobehi and James M. Beach

4.6. Experiment Five

4.6.1. Methods and Materials

All methods and materials are the same as in the previous experiment except for the following.

4.6.1.1. Modification of the fundus camera

Retinal images presented in this chapter were taken by a modified Topcon TRC-50EX fundus camera, with a lens and a c-mount through the vertical path of the camera. The modification enables monochromatic fundus illumination and records over 400–720 nm range of wavelengths. The CCD camera was positioned in the polaroid mounting part. A liquid crystal tunable filter (LCTF) 7-nm bandwidth filter was placed in the path of the illumination.73 An Apogee U47 16-bit digital camera was positioned inside the fundus camera, which can provide low readout noise images. Hyperspectral images were taken through the vertical viewing port by an imaging spectrograph and digital camera (model VNIR 100; Photon Industries Inc., Mississippi Stennis Space Center, USA) across the fundus image (Figs. 4.3 and 4.4).

4.6.1.2. Sodium fluorescein dye injection

In order to enhance the blood vessel appearance in angiograms, a sodium fluorescein dye was injected into the monkey’s arm.Angiograms were acquired 1–2 minutes after the intravenous injection. When the dye circulated through the retinal arteries, capillaries, and veins, it was progressively eliminated from the vasculature, while staining the ONH and lesions. The biomedical image acquisition software developed by HyperVisual with Photon Industries, Mississippi Stennis Space Center, USA assessed the changes in the relative blood oxygen saturation in the retinal ONH vessels (Figs. 4.5 and 4.6). Then, saturation maps were calculated from the oxygenated and deoxygenated hemoglobin spectral signatures.

4.6.1.3. Automatic control point detection

The objective of the adaptive exploratory algorithm is to estimate the initial good-guess of the control points representing the target images’

174

Hyperspectral Image Analysis for Oxygen Saturation Automated Localization of the Eye

salient features.88 The following information is available for the adaptive exploratory algorithm. (1) 2D images’ resolutions and sizes (width: m pixels; height: n pixels). (2) Corresponding binary m × n map with 1 standing for white and 0 standing for black. (3) Edge detection algorithm is available for retinal vessel detection with 1 standing for nonedge pixels and 0 standing for edge pixels. It is assumed that both of the reference and the input images do not have huge rotation, scaling, or translation, and, thus, the same features on each image are close to each other.

A global adaptive threshold developed by Otsu89 is employed to convert the gray level colors to black-and-white. The threshold is a normalized intensity value that lies in the range [0, 1]. Otsu’s method chooses the optimal threshold to minimize the intraclass variance of the black and white pixels and to maximize between-class variance in a grayscale image. The algorithm calculates the statistics of the image itself to set the threshold and use the histogram to choose its value at some percentile as a reference value of region strength. Therefore, Otsu’s method is nonparametric and nonsupervised. Canny edge detector90 is employed to extract the ONH vasculature from the binary images. Canny’s method detects the edges at the zero-crossings of the second directional derivative of the image. In order to make the localization of magnitude maxima accurate, a filter is defined by optimizing a performance index, which enhances real positive and real negative.

The retinal salient features preserved in the Canny edges are the retinal vessel bifurcations, from which the control points are selected using the adaptive exploratory algorithm (Figs. 4.26, 4.27, 4.28, and 4.29). Initial good-guess of the control points ensures fused image generated at an efficient computational time. Bad control point selection will significantly increase the computation cost, or even cause the image fusion fail. Vessels or some particular abnormalities make images not necessarily matching the retina structures. Even when structure and function correspond, the abnormality still happens sometimes if inconsistence exists between structural and functional changes. Furthermore, angiogram images usually have higher resolution and are rich in information, whereas fundus and the oxygen saturation images have lower resolution and are indeed abstract with some details or even missing some small vessels. Practically, those situations are unavoidable and create difficulties in extracting the control points because the delineation of the vein boundaries may not be precise.

175

Bahram Khoobehi and James M. Beach

Fig. 4.26. Angiogram image control point selection. Left — five control points selected by using the adaptive exploratory algorithm; Right — enlarged image of the control point selection at the vessel bifurcation point. Reprinted with permission from Cao, H., and Khoobehi, B. Angiogram, fundus, and oxygen saturation optic nerve head image fusion. In: Farkas, D.L., Nicolau, D.V., and Leif, R.C. (eds.),

Imaging, Manipulation, and Analysis of Biomolecules, Cells, and Tissues VII, Proc of SPIE. 7182, 71821U. ©2009 SPIE.

Fig. 4.27. Fundus color image control point selection. Left — 4 control points selected by using adaptive exploratory algorithm; Right — enlarged image of the control point selection at the vessel bifurcation point. Reprinted with permission from Cao, H. and Khoobehi, B. Angiogram, fundus, and oxygen saturation optic nerve head image fusion. In: Farkas, D.L., Nicolau, D.V., and Leif, R.C. (eds.),

Imaging, Manipulation, and Analysis of Biomolecules, Cells, and Tissues VII, Proc of SPIE. 7182, 71821U. ©2009 SPIE.

For control point detection on the median filtering output, the median filtering is applied toward the black and white ONH oxygen saturation image. Canny edge detection and the adaptive exploratory algorithm are then applied toward the output image after the noise removal for the control point detection.

176

Hyperspectral Image Analysis for Oxygen Saturation Automated Localization of the Eye

Fig. 4.28. ONH oxygen saturation image (left) and its black and white image with median filter applied (right). Reprinted with permission from Cao, H. and Khoobehi, B. Angiogram, fundus, and oxygen saturation optic nerve head image fusion. In: Farkas, D.L., Nicolau, D.V., and Leif, R.C. (eds.),

Imaging, Manipulation, and Analysis of Biomolecules, Cells, and Tissues VII, Proc of SPIE. 7182, 71821U. ©2009 SPIE.

Fig. 4.29. ONH oxygen saturation image’s Canny edge (left) and the control point selection by using the adaptive exploratory algorithm (right). Reprinted with permission from Cao, H. and Khoobehi, B. Angiogram, fundus, and oxygen saturation optic nerve head image fusion. In: Farkas, D.L., Nicolau, D.V., and Leif, R.C. (eds.), Imaging, Manipulation, and Analysis of Biomolecules, Cells, and Tissues VII, Proc of SPIE. 7182, 71821U. ©2009 SPIE.

4.6.1.4. Fused image optimization

An optimization procedure is required to adjust the initial good-guess control points in order to achieve the optimal result. The process can be formulated as a heuristic problem of optimizing an objective function

177

Bahram Khoobehi and James M. Beach

Fig. 4.30. Fused ONH image from angiogram and the fundus color images prior to the optimization (left) and the improved fused image after the iterative optimization (right). Reprinted with permission from Cao, H. and Khoobehi, B. Angiogram, fundus, and oxygen saturation optic nerve head image fusion. In: Farkas, D.L., Nicolau, D.V., and Leif, R.C. (eds.), Imaging, Manipulation, and Analysis of Biomolecules, Cells, and Tissues VII, Proc of SPIE. 7182, 71821U. ©2009 SPIE.

Fig. 4.31. Fused ONH image from angiogram and the oxygen saturation images prior to the optimization (left) and the improved fused image after the iterative optimization (right). Reprinted with permission from Cao, H. and Khoobehi, B. Angiogram, fundus, and oxygen saturation optic nerve head image fusion. In: Farkas, D.L., Nicolau, D.V., and Leif, R.C. (eds.), Imaging, Manipulation, and Analysis of Biomolecules, Cells, and Tissues VII, Proc of SPIE. 7182, 71821U. ©2009 SPIE.

that maximizes the Mutual-Pixel-Count between the reference and input images.91 The algorithm finds the optimal solution by refining the transformation parameters in an ordered way. By maximizing the objective function, one image’s vessels are supposed to be well overlaid onto those of the other image (Figs. 4.30 and 4.31).

Mutual-pixel-count measures the ONH vasculature overlapping for corresponding pixels in both images. It is assumed that the retinal vessels are represented by 0 (black pixel) and background is represented by 1 (white

178