Thomas P. Karnowski et al.

information about ON locations using linear discriminant analysis (LDA). The performance of the PCA-LDA method was shown to be superior to that of PCA alone, but we also investigated ways to combine this method with our vessel-segmentation-based method. In our first improvement, we trained the vessel segmentation method and estimated the performance on the training set to identify false positives where the vessel-based method had issues. These data were then used as part of the “non-ON” population in the LDA training phase. We refer to this as “directing” the PCA-LDA toward the images that give the vessel-based method trouble. This addition is an intuitive and straightforward modification to the method and essentially comes “for free” in that we simply choose different examples of the non-ON areas to train the LDA classifier; our main goal is to prevent the two methods from picking the same wrong location. The new classifiers were then tested on an unseen testing set and some improvement in performance was found Ref. [49].

We use the two complementary ON location methods to estimate the accuracy of the measurement. Our strategy was to measure the distance between the two estimates, and when this measurement exceeded a threshold (confidence level), we rejected the measurement. In practice, this would mean that we would refer the image directly to the oversight physician. In Ref. [50], we reviewed how these two complementary methods (one focusing on the vasculature tree and the other focusing on the ON appearance) could be used in tandem to produce a confidence measurement. We showed empirically, on two data sets, how limiting the number of automatically screened images could improve accuracy by simply thresholding the distance between the estimates of the methods.

14.2.4. Macula Localization

Once the ON is located, the macula is found using the method described in Ref. [47]. This method again uses the vascular tree segmentation and attempts to fit a parabola to the vascular tree as shown in Fig. 14.12. The vascular tree segmentation is “cleaned” slightly by focusing only on the core trunk values by removing branches that are smaller in thickness, which essentially makes the structure look much more like a parabola. The structure is then thinned so that the vessels are represented by a very small

438

Automating the Diagnosis, Stratification, and Management of DR

Fig. 14.12. Example of localizing the macula using parabolic model fit to the vessel structure.

number of pixels. Then, these pixel coordinates are taken and fit to a parabola using the ON estimate and the Marquardt-Levenberg nonlinear least-squares algorithm.51 This method is similar to that of Ref. [15], but, in our case, we only seek to fit the orientation and curvature of the vessel structure, making the problem simpler. Once the orientation is found, we estimate the fovea position by using the mean of the ON–macula distances from a training set of images.

14.2.5. Lesion Segmentation

The two typical lesions that first appear in a patient affected by DR are microaneurysms and hemorrhages.While our system is intended to diagnose

439

Thomas P. Karnowski et al.

other retinal diseases, including age-related macular degeneration (AMD), the main driver is DR, and consequently, our initial focus is on these types of lesions. Microaneurysms are focal dilatations of retinal capillaries from 10 to 100 microns in diameter that appears as small red dots in a fundus image. Technically, they are not retinal lesions. However, we tend to group them in this category, because they are a phenomena associated with disease (as opposed to physiology) that we would like to detect. Hemorrhages are due to the leakage of blood from the wall of a damaged capillary or microaneurysm. These lesions are the most challenging to detect, but are the ones that allow the earliest disease detection. Exudates are yellowish in appearance and are sharp and bright structures caused by fluid leakage. They often have a circinate pattern of appearance regarding the entire population.52

There have been many approaches to lesion segmentation attempted in the literature. This work has been summarized in Refs. [10,53]. Some more recent approaches of interest for microaneurysms, exudates, and drusen (which appear similar to exudates but are more indicative of diseases such as AMD) include, but are not limited to Refs. [54–61]. An ongoing project of special note is the Retinopathy Online Challenge, which uses a publicly available database and evaluation method for algorithm comparison.62

In our work, we have developed a new algorithm for the segmentation of microaneurysms.63 The algorithm is based on two steps, a background removal process that employs wavelets and the actual microaneurysms detection. The microaneurysm detection is performed with a new method, the Radon Cliff operator. Making use of the Radon transform, the operator is able to detect single noisy Gaussian-like circular structures regardless of their size or position in a window. This method has several advantages over existing microaneurysms detectors: the size of the lesions can be unknown, it automatically distinguishes lesions from the vasculature in general, and it provides fair microaneurysm localization without post-processing the candidates with machine-learning techniques, facilitating the training phase. Figure 14.13 shows an example of the current algorithm output.

Exudates are another visible sign of DR and a marker for the presence of retina edema. As mentioned, various authors have developed segmentation algorithms for the automated detection of DR, however, to our knowledge, none has explicitly addressed the problem of analyzing a retina with a high

440

Automating the Diagnosis, Stratification, and Management of DR

Fig. 14.13. Microaneurysm detection using Radon Cliff operator.

degree of reflecting artifacts due to the nerve fiber layer (NFL). This structure is particularly visible in young patients, especially on dark pigmented retinas such as seen in African American patients. We have developed a technique to detect bright lesions in patients with a high degree of reflective NFL. First, the candidate bright lesions are detected using image equalization and histogram analysis. Then, a classifier is trained using texture descriptors (multi-scale local binary patterns) and other statistical features in order to remove the false positives in the lesion detection. Finally, the area of the lesions is used to diagnose DR. More details are covered in Ref. [64]. Currently, the algorithm has been refined using a wavelet background subtraction approach similar to the one proposed for the microaneurysms segmentation. Figure 14.14 shows an example of this segmentation.

14.2.6. Overall Fundus Description and Stratification

Our overall fundus description consists of a measurement of 170 features related to the lesions detected (such as histograms of the sharpness of the lesion edges, their intensity, shape properties, etc.), the vascular density

441

Thomas P. Karnowski et al.

Fig. 14.14. Exudate detection using wavelet analysis and texture descriptors.

within the lesion population, population moments, and invariant moments relative to the central macula. Our goal is to determine an index that can be used to locate similar imagery by comparison. This determination can be made if we can produce an index that provides discrimination between the different disease states of interest in the archive (which were ground-truthed by the oversight physician in the process of building the archive). We use linear discriminant analysis (LDA) to project our image features in 170dimensional space onto a smaller subspace which ideally optimally discriminates between our defined disease (and normal) disease states. For this discussion, we are applying a traditional multiple LDA method based on the well-known Fischer discriminant.65

To perform retrievals from the archive using the image index, we determine a similarity between a query image that has also been subjected to feature extraction through lesion segmentation and lesion population feature extraction. When the archive becomes very large, fast, and efficient methods for searching must be used. However, in our initial stage of development, our database size is still small and therefore, the indexing methods are not required.

We have described our CBIR system and its performance on test data sets in Refs. [66, 67] and summarize these key results here for completeness. The developed CBIR method uses the retrieval response to our query image, represented by an index vector q, to estimate the posterior probability, P(ωi|v), of each defined disease state ωi. The retrieval process is similar to a k-nearest neighbor (k-NN) method.65 Nearest neighbor classifiers function by locating the population of labeled data points nearest to an unknown point in index space for a specified number of neighbors, k.

442

Automating the Diagnosis, Stratification, and Management of DR

A posterior probability can be derived from a simple k-NN voting scheme or, as in our case, a weighted summation of similarities can be used.

In a simple k-NN estimate, the posterior probability is expressed as P(ωi|v) = ki/ k, where ki represents the number of neighborhood vectors corresponding to disease state ωi of the k neighbors retrieved. For our method, we use a weighted summation of the similarity between points to give,

where the exponent, w ≥ 0, increases the influence of the points closest to the query point. The estimate approaches a nearly optimal posterior estimate, as the number of records in the system increases, meaning the diagnostic performance of the archive will theoretically improve as the archive population increases.

Note that we have also incorporated a confidence value, σ, using Poisson statistics.66 Poisson statistics can be applied to phenomena of a discrete nature, such as the rate of disease occurrence in patients. By using the property that the standard deviation in the sampling of the disease category, ωi, is the square root of the number of counts in the category, we can estimate our confidence in the posterior probability of a particular disease state as indicated.

We review our results from Refs. [66, 67] here. The authors of these sources used two independent sets of image data. The first set is an image archive of 1,355 macula-centered images obtained from a DR-screening program in the Netherlands,68,69 which we abbreviate as NL. The second image set of 98 images was used for comparison purposes and originated from a Canadian Native American population.70 We designate them as C.

Table 14.2 shows consolidated results for a number of different trials in which the two parameters of image quality and posterior probability confidence were varied for the independent data set. In the first row describing the performance of the patient archive containing 1,355 records, we have applied a statistical hold-one-out (HOO) procedure to determine the expected performance of the system based on the internal consistency of the data. The HOO performance values of sensitivity and accuracy are listed for the archive as 90%, and 95%, respectively. Since HOO performance often presents slightly higher results than is generally noted from truly

443