5.5Summary

In this chapter, we have described our work using CBIR and retinal image databases in a regional ocular telehealth network. The network permits remote diagnosis of DR using high-throughput methods to meet the growing need for highthroughput disease assessment and management. The network infrastructure for automated diagnosis of DR provides a method for low-cost, realtime diagnosis and patient referral in the primary care environment. We also describe the design of the underlying network infrastructure, which emphasizes high-speed data transmission for real-time image analysis, secure data encryption, and cost-effective implementation and transmission of protected health information to meet Federal HIPAA compliance regulations.

Acknowledgments These studies were supported in part by grants from Oak Ridge National Laboratory, the National Eye Institute, (EY017065), and the Health Resources and Services Administration, by an unrestricted UTHSC departmental grant from Research to Prevent Blindness, New York, NY and by the Plough Foundation, Memphis, TN.

References

1.Bartels PH, Wied GL (1981) Automated image analysis in clinical pathology. Am J Clin Pathol 75(3 Suppl):489–493

2. Byng JW, Boyd NF, Fishell E, Jong RA, Yaffe MJ (1996) Automated analysis of mammographic densities. Phys Med Biol 41:909–923

3. Katsuragawa S, Doi K (2007) Computer-aided diagnosis in chest radiography. Comput Med Imaging Graph 31(4–5):212–223

4. Prevention of blindness from diabetes mellitus: report of a WHO consultation. World Health Organization. Available at http://www.who.int/diabetes/en. Accessed on 2006

5.International clinical classification of diabetic retinopathy, severity of diabetic macular edema. International Council of Ophthalmology. Available at http://www.icoph.org/enhancing_eyecare/interna-

tional_clinical_guidelines.html. Accessed on 2011 6. Pugh JA, Jacobson JM, Van Heuven WA, Watters JA,

Tuley MR, Lairson DR, Lorimor RJ, Kapadia AS, Velez R (1993) Screening for diabetic retinopathy. The wideangle retinal camera. Diabetes Care 16(6):889–895

7. Javitt JC, Aiello LP, Bassi LJ, Chiang YP, Canner JK (1991) Detecting and treating retinopathy in patients

with type I diabetes mellitus: savings associated with improved implementation of current guidelines. Ophthalmology 98(10):1565–1573

8. Harris MI, Flegal KM, Cowie CC, Eberhardt MS, Goldstein DE, Little RR, Wiedmeyer HM, Byrd-Holt DD (1998) Prevalence of diabetes, impaired fasting glucose, and impaired glucose tolerance in U.S. Adults. The third national health and nutrition examination survey, 1988–1994. Diabetes Care 21(4):518–524

9.Lin DY, Blumenkranz MS, Brothers RJ, Grosvenor DM (2002) The sensitivity and specificity of singlefield nonmydriatic monochromatic digital fundus

photography with remote image interpretation for diabetic retinopathy screening: a comparison with ophthalmoscopy and standardized mydriatic color photography. Am J Ophthalmol 134:204–213

10. Rudnisky CJ, Tennant MTS, Weis E, Ting A, Hinz BJ, Greve MDJ (2007) Web-based grading of compressed stereoscopic digital photography versus standard slide film photography for the diagnosis of diabetic retinopathy. Ophthalmology 114:1748–1754

11. Wei J, Valentino D, Bell D, Baker R (2006) A webbased telemedicine system for diabetic retinopathy screening using digital fundus photography. Telemed J E Health 12:50–57

12.Cavallerano AA, Cavallerano JD, Katalinic P, Tolson AM, Aiello LP, Aiello LM (2003) Joslin Vision Network Clinical Team. Use of Joslin Vision Network digital-video nonmydriatic retinal imaging to assess diabetic retinopathy in a clinical program. Retina

23:215–223

13. Li Y, Karnowski TP, Tobin KW, Giancardo L, Morris S, Sparrow SE, Garg S, Fox K, Chaum E (2011). A health insurance portability and accountability act-compliant ocular telehealth network for the remote diagnosis and management of diabetic retinopathy. Telemed J E Health 17(8):627–634

14.Teng T, Lefley M, Claremont D (2001) Progress towards automated diabetic ocular screening: a review of image analysis and intelligent systems for diabetic

retinopathy. Med Biol Eng Comput 40:2–13

15. Fleming AD, Philip S, Goatman KA, Olson JA, Sharp PF (2006) Automated assessment of diabetic retinal image quality based on clarity and field definition. Invest Ophthalmol Vis Sci 47(3):120–1125

16.Niemeijer M, Abramoff MD, van Ginneken B (2006) Image structure clustering for image quality verifica-

tion of color retina images in diabetic retinopathy screening. Med Image Anal 10(6):888–898

17. Usher DB, Himaga M, Dumskyj MJ (2003) Automated assessment of digital fundus image quality using detected vessel area. In: Proceeding of medical image understanding and analysis, British Machine Vision Association (BMVA), Sheffield, 2003, pp 81–84

18. Lalonde M, Beaulieu M, Gagnon L (2001) Fast and robust optic disc detection using pyramidal decomposition and Hausdorff-based template matching. IEEE Trans Med Imaging 20(11):1193–1200

19.Giancardo L, Abramoff MD, Chaum E, Karnowski TP, Meriaudeau F, Tobin KW (2008) Elliptical local

vessel density: a fast and robust quality metric for fundus images. Conf Proc IEEE Eng Med Biol Soc 35:34–37

20.Giancardo L, Meriaudeau F, Karnowski TP, Chaum E and Tobin K (2010) Quality assessment of retinal fun-

dus images using elliptical local vessel density. In New Developments in Biomedical Engineering, Ed. Domenico Campolo, INTECH. http://sciyo.com/ books/show/title/new-developments-in-biomedical- engineering

21.Chaudhuri S, Chatterjee S, Katz N, Nelson M, Goldbaum M (1989) Detection of blood vessels in retinal images using two-dimensional matched filters. IEEE Trans Med Imaging 8(3):263–269

22.Hoover A, Kouznetsova V, Goldbaum M (2000) Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response. IEEE Trans Med Imaging 19(3):203–210

23.Jiang X, Mojon D (2003) Adaptive local thresholding by verification-based multithreshold probing with application to vessel detection in retinal images. IEEE Trans Pattern Anal Mach Intell 25(1):131–137

24.Chapman N, Witt N, Gao X, Bharath AA, Stanton AV, Thom SA, Hughes AD (2001) Computer algorithms for the automated measurement of retinal arteriolar diameters. Br J Ophthalmol 85(1):74–79

25. Wang L, Bhalerao A, Wilson R (2007) Analysis of retinal vasculature using a multiresolution Hermite model. IEEE Trans Med Imaging 26:137–152

26. Niemeijer M, Staal J, van Ginneken B, Loog M, Abramoff MD (2004) Comparative study of retinal vessel segmentation methods on a new publicly available database. Proc SPIE v5370:648–656

27. Fang B, Hsu W, Lee ML (2003) Reconstruction of vascular structures in retinal images. In: Proceedings of the international conference on image processing (ICIP’03), vol. II, Barcelona, 2003, pp 157–160

28.Zana F, Klein J (2001) Segmentation of vessel-like patterns using mathematical morphology and curvature evaluation. IEEE Trans Image Proc 10:1010–1019

29.Foracchia M, Grisan E, Ruggeri A (2004) Detection of the optic disk in retinal images by means of a geometrical model of vessel structure. IEEE Trans Med Imaging 23:1189–1195

30.Hoover A, Goldbaum M (2003) Locating the optic nerve in a retinal image using the fuzzy convergence of the blood vessels. IEEE Trans Med Imaging 22:951–958

31.Niemeijer M, Abramoff MD, van Ginneken B (2007) Segmentation of the optic disc, macula and vascular arch in fundus photographs. IEEE Trans Med Imaging 26(1):116–127

32.Niemeijer M, Abramoff MD, van Ginneken B (2009) Fast detection of the optic disc and fovea in color fundus photographs. Med Image Anal 13:859–870

33.Youssif A-R, Ghalwash AZ, Ghoneim A-R (2008) Optic disc detection from normalized digital fundus images by means of a vessels’ direction matched filter. IEEE Trans Med Imaging 27(1):11–18

34.Karnowski TP, Aykac D, Chaum E, Giancardo L, Li Y, Tobin KW Jr, Abramoff MD (2009) Practical consid-

erations for optic nerve location in telemedicine. Conf Proc IEEE Eng Med Biol Soc 1:6205–6209

35.Tobin KW, Chaum E, Govindasamy VP, Karnowski TP (2007) Detection of anatomic structures in human retinal imagery. IEEE Trans Med Imaging 26:1729–1739

36.Li H, Chutatape O (2004) Automated feature extraction in color retinal images by a model based approach. IEEE Trans Biomed Eng 51(2):246–254

37.Karnowski TP, Govindasamy VP, Tobin KW, Chaum E (2006) Locating the optic nerve in retinal images: comparing model-based and Bayesian decision methods. Conf Proc IEEE Eng Med Biol Soc 1:4436–4439

38.Karnowski TP, Aykac D, Chaum E, Giancardo L, Li Y, Tobin KW, Abràmoff MD (2009) Evaluating the accuracy of optic nerve detections in retina imaging by using complementary methods. In: Proceedings of world congress of medical physics and biomedical engineering, Munich, 2009

39.Marquardt D (1963) An algorithm for least-squares estimation of nonlinear parameters. SIAM J Appl Math 11:431–441

40.Patton N, Aslam TM, MacGillivray T, Deary IJ, Dhillon B, Eikelboom RH, Yogesan K, Constable IJ (2006) Retinal image analysis: concepts, applications and potential. Prog Retin Eye Res 25:99–127

41.Niemeijer M, van Ginneken B, Cree MJ, Mizutani A et al (2010) Retinopathy online challenge: automatic detection of microaneurysms in digital color fundus photographs. IEEE Trans Med Imaging 29(1):185–195

42. Giancardo L, Meriaudeau F, Karnowski TP, Chaum E, Tobin KW, Li Y (2010) Microaneurysms detection with the Radon Cliff Operator in retinal fundus images. In: Proceedings of SPIE medical imaging, San Diego, 2010

43.Giancardo L, Chaum E, Karnowski TP, Meriaudeau F, Tobin KW, Li Y (2009) Bright retinal lesions detection using color fundus images containing reflective features. In: Proceedings of world congress of medical physics and biomedical engineering, Munich, 2009

44.Chaum E, Karnowski TP, Govindasamy VP, Abdelrahamen M, Tobin KW (2008) Automated diagnosis of retinopathy by content-based image retrieval. Retina 28:1463–1477

45.Tobin KW, Abramoff MD, Chaum E, Giancardo L, Govindasamy VP, Karnowski TP, Tennant MTS (2008) Using a patient image archive to diagnose retinopathy. Conf Proc IEEE Eng Med Biol Soc 1:5441–5444

46.Tobin KW, Abdelrahman M, Chaum E, Govindasamy VP, Karnowski TP (2007) A probabilistic framework for content-based diagnosis of retinal disease. Conf

Proc IEEE Eng Med Biol Soc 2007:6744–6747

47. Duda RO, Hart PE, Stork DG (2001) Pattern classification, 2nd edn. Wiley, New York

48.Abramoff MA, Suttorp-Schulten M (2005) Webbased screening for diabetic retinopathy in a primary care population: the eyecheck project. Telemed J E Health 11:668–674

49.Abramoff MA, Veirgever M, Neimeijer M, Russel S, Suttorp-Schulten M, Van Ginneken B (2008)

Evaluation of a system for automatic detection of diabetic retinopathy from color fundus photographs in a large population of patients with diabetes. Diabetes Care 31:193–198

50.Niemeijer M, van Ginneken B, Staal J, SuttorpSchulten MS, Abramoff MD (2005) Automatic

detection of red lesions in digital color fundus photographs. IEEE Trans Med Imaging 24:584–592

51.Niemeijer M, Abramoff MD, van Ginneken B (2009) Information fusion for diabetic retinopathy CAD in digital color fundus photographs. IEEE Trans Med Imaging v28:775–785

6.1Introduction

Diabetic retinopathy (DR) is the leading cause of low vision and blindness in people of working age in Europe and United States and the more common microvascular complication of diabetes. It is also projected that during the next 20–30 years, the

J. Cunha-Vaz ( ) • R. Bernardes

AIBILI – Association for Innovation and Biomedical Research on Light and Image, Coimbra, Portugal

Faculty of Medicine, University of Coimbra, Centre of New Technologies for Medicine,

Coimbra, Portugal

e-mail: cunhavaz@aibili.pt

T. Santos • L. Ribeiro

AIBILI – Association for Innovation and Biomedical Research on Light and Image, Centre of New Technologies for Medicine, Coimbra, Portugal

C. Oliveira

Critical Health, Centre for Clinical Trails, Coimbra, Portugal

C. Lobo

AIBILI – Association for Innovation and Biomedical Research on Light and Image, Centre of New Technologies for Medicine, Coimbra, Portugal

Faculty of Medicine, University of Coimbra,

Coimbra, Portugal

Department of Ophthalmology, University Hospital

of Coimbra, Coimbra, Portugal

I. Pires

AIBILI – Association for Innovation and Biomedical Research on Light and Image Centre for Clinical Trails, Coimbra, Portugal

Department of Ophthalmology, University Hospital

of Coimbra, Coimbra, Portugal

number of persons affected with diabetes mellitus will increase by as much as 35% [1]. It is well recognized from clinical experience that the evolution and progression of DR varies between different individuals independently of the duration of the disease and the status of its metabolic control. Diabetic patients with similar levels of chronic hyperglycaemia do not develop necessarily the same DR complications, and not every patient develops macular oedema or proliferative retinopathy, the complications associated with vision loss.

There is now accumulated evidence indicating that only the non-proliferative stage of DR (NPDR) is directly due to the systemic disease and associated hyperglycaemia. Macular oedema and proliferative diabetic retinopathy are late complications of diabetic retinopathy. Macular oedema is a direct result of a widespread alteration of the blood-retinal barrier, and proliferative retinopathy occurs only after the development of large areas of capillary closure with the ensuring ischaemia. Neovascularization in DR is the direct result of the ischaemia and when established is not influenced by the diabetic metabolic control. Its course and management are not different from other situations in the retina where neovascularization develops such as retinal vein occlusion [2].

Diabetic retinal lesions are still reversible at the initial stages of mild NPDR, before the complications of DR, macular oedema and proliferative retinopathy occur. It is this stage of the disease that needs to be well characterized if we want to stop disease progression and improve management of DR.

Four main alterations characterize the early stages of DR: microaneurysms/haemorrhages, alteration of the blood-retinal barrier, capillary closure and alterations of the neuronal and glial cells of the retina. These alterations may be monitored by microaneurysms counting methodologies and retinal thickness measurements. A combination of these methods using novel methods of multimodal imaging of the retina has contributed to the identification of three different phenotypes of NPDR, showing different patterns of disease progression: phenotype A, including eyes with little abnormal leakage, a slow rate of microaneurysm formation and no signs of capillary closure; phenotype B, including eyes characterized by persistently high leakage values and increased retinal thickness measurements, variable rates of microaneurysm formation and no signs of capillary closure (in this phenotype, the alteration of the blood-retinal barrier is the dominant feature); and phenotype C, including eyes with variable leakage and variable retinal thickness values, high rates of microaneurysm formation and disappearance and clear signs of capillary closure. This third phenotype shows a clear predominance of capillary closure and early development of retinal ischaemia. Long-term follow-up of these different groups of eyes/patients for a period of 7 years showed that only eyes/patients belonging to phenotypes B and C developed clinical significant macular oedema with clear indication for photocoagulation treatment according to ETDRS guidelines. None of the eyes identified initially as phenotype A developed after 7 years of follow-up severe macular oedema needing laser photocoagulation. In summary, the phenotype A is characterized by lack of progression, suggesting that this phenotype has a slow evolution without development of the characteristic complications of NPDR, macular oedema and proliferative retinopathy, at least during a period of 7 years.

On the other hand, the other DR phenotypes, the leaky type or phenotype B and the ischaemic type or phenotype C, lead much more frequently to the development of severe macular oedema with incidences at 7 years of 41% and 50%, respectively [3].

The characterization of these three different phenotypes of NPDR confirms the general clinical impression that the evolution and progression of DR varies between different individuals.

6.2Automated Monitoring

of Retinopathy Progression: Microaneurysm Turnover

It is, therefore, of fundamental importance to monitor the progression of the disease in a specific patient and identify if he is a ‘progressor’, i.e. a patient that shows signs of rapid progression and to which phenotype of progression he belongs. Some eyes/patients need special attention and timely intervention to avoid development of the DR complications, macular oedema or proliferative DR.

The major alterations that occur in NPDR and need to be monitored are microaneurysms dynamics, namely, their formation and disappearance, vascular leakage with subsequent oedema and hard exudates formation and capillary closure.

Visual function loss occurs characteristically late in DR because the eye has a large functional reserve of vision, and DR affects initially the inner layers of the retina away from the photoreceptors. Therefore, structural changes are detected in DR earlier than functional changes. We have, therefore, to focus on evidence of structural changes if we want to follow progression in the earliest stages of DR.

One of the best candidates for non-invasive imaging of the eye fundus is clearly fundus digital photography because retinal cameras are widely available, and the data obtained may be supported and enhanced by computer-assisted procedures.

To identify progression it is essential to collect sequential series of images, and these images must be compared. The need for co-registration of these sequences of images is, therefore, of great relevance. By applying novel co-registration procedures and automated comparative analysis software, it is now possible to perform reliable sequential comparisons of fundus digital photography images.

The RetmarkerDR is a software now available (Critical Health, Portugal) which is able to automatically detect changes occurring in eye fundus digital images, by comparing successive visits to the reference images, in each eye, based on co-registration and co-localization of the changes (Figs. 6.1 and 6.2).

On fundus photography, microaneurysms and small haemorrhages are the initial changes detected in the diabetic retina. They may be

counted, and retinal microaneurysm counting has been suggested as an appropriate marker of retinopathy progression [4, 5].

Retinal microaneurysms are important lesions of diabetic retinopathy, and even one or two microaneurysms in an eye should not be regarded as unimportant [6]. When examining 1,809 patients in the UKPDS cohort that had either no retinopathy or microaneurysms only at entry, they showed that the number of microaneurysms

had a high predictive value for worsening retinopathy at 3, 6, 9 and 12 years after entry into the study [6]. Similar findings had been presented by Klein et al. who looked at the relationship of retinal microaneurysms to the progression of diabetic retinopathy over a 4-year period [7]. In their study, the number of microaneurysms at the baseline examination was positively associated with significant progression of retinopathy.

More recently, Sjolie et al. showed that microaneurysms counts were predictive of an increased risk of retinopathy [8].

However, our research has shown that the total number of microaneurysms detected in colour fundus photographs offers lower sensitivity in detecting progression of the retinopathy when compared with the determination of microaneurysms turnover, taking into account the exact location of new microaneurysms in successive fundus photographs taken at 6-month or 1-year intervals, apparently because the regressed microaneurysms are constantly balanced by the new ones [9].

We found that differences between successive visits using microaneurysm counts are less reliable than microaneurysm formation rates, which take into account newly formed microaneurysms and give more accurate information on ‘activity of the retinopathy’. Furthermore, we have also found much better agreement between graders when determining microaneurysm turnover [10].

Recently, Sharp et al. [11] found that the microaneurysm turnover varied widely between eyes of the same retinopathy level. This is also consistent with our findings. Microaneurysm turnover has been shown in our studies to vary between eyes that were classified with the same retinopathy level. Particularly relevant and of major interest is the finding that the patients who have higher microaneurysm turnover values are the ones that go on to develop clinically significant macular oedema (CSME) and show a more rapid retinopathy progression, particularly in association with poor metabolic control demonstrated by higher HbA1c values. Microaneurysm turnover appears to be a distinctive characteristic that indicates activity of disease and rapid progression in eyes with apparently similar retinopathy level.

The observation that in the group with diabetes type 2, the level of metabolic control, given by HbA1c values, correlates with retinopathy progression confirms previous reports [12]. It is interesting that other systemic variables, such as blood pressure and blood-lipid levels, did not appear to be relevant in this relatively wellcontrolled group of patients.

Microaneurysms are the key lesion in the early stages of DR, and our work demonstrates consistency in the demonstration of microaneurysm turnover values [10]. Our studies demonstrate that it is not the absolute total number of microaneurysms at a certain point in time that may provide the best indication of retinopathy progression, but the rate of microaneurysm turnover in successive visits during a 1- or 2-year period.

We have shown that it is possible to use microaneurysm computed from non-invasive colour fundus photographs as a biomarker to identify eye/patients at risk of progression for CSME. A microaneurysm formation rate of at least two microaneurysms/eye in eyes with mild NPDR and diabetes type 2 appears to identify patients at risk for progression to CSME as well. In one recent and larger study, with a 10-year follow-up of 113 eyes/patients, the percentage of false negatives (eyes that developed CSME with a low microaneurysm formation rate) was 29.4% (5/17), and the percentage of false positives (eyes that did not develop CSME with a high microaneurysm formation rate) was 8.3% (8/96), resulting in a sensitivity in predicting CSME development of 70.6%. The high negative predictive value for CSME (94.6%, 88/93) indicates that a low microaneurysm turnover, i.e. less than two microaneurysms/ year, identifies particularly well the eyes/patients that are not expected to progress to CSME within a 10-year period.

In another study, a group of 290 eyes that were followed by fundus photography during a period of 5 years in the Caldiret study, coordinated by Munich University, it was possible to compare 49 eyes that did develop CSME over the period of the study with 241 eyes that did not develop CSME in the same 5-year period (Ulbig, M., et al., personal communication 2011).