Kluwer - Handbook of Biomedical Image Analysis Vol

Figure 14.2: Segmentation of the dual-echo MR image using training. The left image shows user-defined training regions for the different tissue classes. The corresponding tessellation of the feature space (spanned by the joint intensity distribution) is shown in the middle, resulting in the segmentation on the right.

The applied procedure can be regarded as a generalized thresholding, aiming at the identification of areas in a feature space, i.e. in the two-dimensional intensity distribution (Fig. 14.1(b)), which uniquely characterize the different tissue classes (as gray or white matter of the brain). These areas are usually determined during a training phase, where the user identifies examples for each tissue class (e.g. in the form of regions of interest as illustrated in Fig. 14.2(a)). Standard pattern recognition procedures (e.g., as k-nearest neighbor classification) [2] can be used to derive a corresponding tessellation of the feature space (Fig. 14.2(b)) leading to the classification of the entire image (Fig. 14.2(c)).

The success of the segmentation basically depends on the assumption that tissue classes can perfectly be separated in the feature space provided by the measurements. Beside physiologically induced overlaps between features of different tissue classes, limitations of the acquisition process can seriously compromise the efficiency of the method.

The most important sources of error are the presence of noise, the spatial inhomogeneity of the signal intensity generated by the tissue, and the limited spatial resolution of the images leading to partial volume effects.

The presence of voxels containing several tissue classes can be smoothly incorporated into the classification framework by extending the original scheme by mixed tissue classes [3, 4]. As classical methods of noise reduction are based on linear low-pass filtering, they necessarily blur the boundary between different tissues, leading to artificially created partial volume effects. Nonlinear

techniques based on anisotropic diffusion processes [5], which selectively stop the smoothing process at spatial positions with large intensity gradients, have been established during the past decade as effective tools for noise reduction, while preserving anatomical structures at tissue boundaries.

Several techniques have been developed for the correction of the spatial intensity bias resulting, for example, from the inhomogeneity of the RF field during MR image acquisition. One possibility considered is the implementation of bias-field correction as a preprocessing step, using generic assumptions about the expected distortions [6, 7]. As an alternative, expectation maximization has been proposed as an iterative framework to perform classification and bias correction simultaneously [8].

One important limitation of the above intensity-based classification framework is that it handles pixels in the image completely independently. This means that the result of the segmentation is invariant to the actual positions of the voxels in the image. This assumption is of course highly nonrealistic as intensities of nearby voxels are usually strongly correlated. This correlation between single pixels can explicitely be described by spatial interaction models. Spatial correlation between the single pixels can be introduced using more or less complex interaction models as, for example, Markov random fields [9, 10] and integrated into the classification framework. As an alternative, postprocessing techniques can be used to correct for erroneous classification. One popular technique is based on mathematical morphology [11], which allows the identification and correction of wrongly classified pixels based on morphological criteria, like the presence of very small, isolated tissue clusters [3]. The latter process is illustrated by the identification of the brain mask on a neuroradiological MR slice Fig. 14.3.

14.3Knowledge-Based Automatic Segmentation

Even the most sophisticated preand postprocessing techniques cannot, however, overcome the inherent limitation of the basically intensity-based methods, namely the assumption that segmentation can be carried out solely based on information provided by the actual image. This assumption is fundamentally

Figure 14.3: Brain segmentation based on morphological postprocessing. Image (a) shows the result of thresholding, which has been eroded (b) in order to break up unwanted connections between different organs. Brain tissue has been identified by connected component labeling (c) and has been dilated back to its original extent (d).

wrong, and the radiologist uses a broad range of related knowledge on the field of anatomy, pathology, physiology, and radiology in order to arrive at a reasonable image interpretation. The incorporation of such knowledge into the algorithms used is therefore indispensable for automatic image segmentation.

Different procedures have been proposed in the literature to approach the problem of representation and usage of prior knowledge for image analysis. Because of the enormous complexity of the necessary prior information, classical methods of artificial intelligence as the use of expert systems [12, 13] can offer only limited support to solve this problem.

14.3.1 Segmentation Based on Anatomical Models

During the past few years, the usage of deformable anatomical atlases has been extensively investigated as an appealing tool for the coding of prior anatomical information for image interpretation. The method is based on a representative deterministic [14] or probabilistic [15] image volume as an anatomical model. For this the actual patient data has to be spatially normalized, thus it has to be mapped onto the template that conforms to the standard anatomical space used by the model. The applied registration procedures range from simple parametric edge matching [16] and rigid registration methods over to increasingly more complex algorithms using affine, projective, and curved transformations. Other methods use complex physically inspired algorithms for elastic deformation or viscous fluid motion [17]. In the latter the transformations are constrained to

be consistent with the physical properties of deformable elastic solids or those of viscous fluids. Viscous fluid models are less constraining than linear elastic models and allow long-distance, nonlinear deformations of small subregions. In these formulations, the deformed configuration of the atlas is usually determined by driving the deformation using only pixel-by-pixel intensity similarity between the images if a reasonable level of automation has to be achieved. Common to all registration methods is the resulting dense vector field that defines the mapping of the subject’s specific anatomy onto the anatomy template used for the atlas.

The usage of deformable atlases seems to be a very elegant way to use prior anatomical information in segmentation, as it allows to gain support from the success of current image registration research. Once the spatial mapping between the atlas and the individual data has been established, it can be used to transfer all spatially related information predefined on the atlas (as organ labels, functional information, etc.) to the actual patient image.

This approach is, however, fundamentally dependent on the anatomical and physiological validity of the generated mapping. It has to be understood, that a successful warping of one dataset into the other, does not guarantee that it also makes sense as an anatomical mapping. In other words, the fact that the registration result looks perfect offers no guarantee that it makes sense from the anatomical point of view. To warp a leg into a nose is perfectly possible, but will not allow any reasonable physiological interpretation.

To make the results of the registration sensible, i.e., useful for image segmentation, one has to solve the correspondence problem. This means that we have to ensure that the mapping establishes a correspondence between the atlas and the patient, which is physiologically and anatomically meaningful. For the time being, purely intensity driven registration cannot be expected to do so in general. Therefore, in the practice such correspondence usually has to be strongly supported using anatomical landmarks [18, 19]. Landmark identification needs, however, in most cases tedious manual work, compromising the quest for automatic procedures. The following section discusses one very popular way to address some of the mentioned fundamental problems of the atlas-based representation of anatomical knowledge. It can, however, hardly be expected that any of the individual methods alone can successfully deal with all aspects of automatic segmentation, and first attempts to combine different approaches have already been published [20].

14.3.2 Segmentation Based on Statistical Models

Anatomical structures show a natural variation for different individuals (interindividual) and also for the same individual (intraindividual) over time. Obvious examples for intraindividual variation of organ shape are the lungs or the heart that both show cyclic variation of their shape. In contrast the bladder shows noncyclic shape variations that mainly depend on its filling. Several researchers have proposed to model the natural (large, but still strongly limited) variability of inter- as well as intraindividual organ shapes in a statistical framework and to utilize these statistics for model-based segmentation. The idea is to code the variations of the selected shape parameters in an observed population (the training set) and characterize this in a possibly compact way. This approach overcomes the limitations of the basically static view of the anatomy provided by the altases from the preceding section.

Such methods fundamentally depend on the availability of parametric models suitable to describe biological shapes. These frameworks always consider variation of shape, but may also include other characteristics, such as the variation of intensity values. Several methods have been proposed for such parametric shape descriptions, as deformable superquadrics augmented with local deformation modeling [21, 22], series expansions [23, 24], or simply using the coordinates of selected organ surface points as used by Cootes in [25] for his point distribution models.

To model these statistics the a priori probability p(p) of a parameter vector p and eventually the conditional probability p(p | D) under the condition of the sensed data D are estimated by a set of samples. Estimation of the probability p(p), however, requires that the entities of the sample set are described in the same way. If, for example, the parameter vector p simply consists of voxel coordinates then it is essential, that the elements of the parameter vectors of the different entities always describe the position of the same point on the different entities at the same position in the vector. To find these corresponding points on the different realizations is an important prerequisite for the generation of statistical shape models. However, it proves difficult, especially as there in no real agreement on what correspondence exactly is, and how it can be measured.

Correspondence can be established in two ways, either discrete or continuously. In the discrete case the surfaces are represented as point sets and the correspondence is defined by assigning points in different point sets to each

other. In the continuous case, parameterizations of the surfaces are defined such that the same parameter values parameterize corresponding positions on the surface.

Looking at the discrete case the most obvious method is to define correspondences manually. To do so a number of landmarks need to be defined on each sample by the user. This method has been successfully used by [26]; however, this technique clearly requires extensive user input, making it only suitable when very limited number of points is regarded. Another possibility when dealing with discrete point sets offers the softassign Procrustes matching algorithm as described by [27]. The algorithm tackles the problem of finding correspondences in two point sets using the Procrustes distance shape similarity measure [18] that quantifies shape similarity.

The most common approach for the approximation of continuous correspondences in 2D is arc-length parameterization. Thus, points of the same parameter on different curves are then taken to be corresponding. This approach heavily depends on the availability of features and is thus bound to fail if the same features can not be located in both modalities. An other interesting view on continuous correspondences in 2D is given by [28], who defines correspondence between closed curves C1 and C2 as a subset of the product space C1 × C2. Kotcheff and Taylor presents in [29] an algorithm for automatic generation of correspondences based on eigenmodes. In [30] Kelemen et al. shows a straight forward expansion of arc-length parameterization based correspondence of curves to establish correspondences on surfaces. They establish correspondence by describing surfaces of a training set using a shape description invariant under rotation and translation presented in [24].

Once the parameterization is selected, the anatomical objects of interest are fully described (at least from the point of view of the envisioned segmentation procedure) by the resulting parameter vector p = { p1, p2, . . . , pn}, where n can of course be fairly large for complex shapes. Possible variations of the anatomy can be precisely characterized by the joint probability function of the shape parameters pi, information of which can be integrated into a stochastic Bayesian segmentation framework as a prior utilizing the knowledge gained from the training data for constraining the image analysis process [22, 31]. It has to be, however, realized that the usually very limited number of examples in the training set forces us to very strongly limit the number of parameters involved in a fitting procedure. A very substantial reduction of the number of parameters can

be achieved based on the fact that the single components of the vector p are usually highly correlated. A simplified characterization of the probability density is possible based on the firstand second-order moments of the distribution (for a multivariate Gaussian distribution this description is exact). The resulting descriptors are

the mean shape:

N

p = 1 p j , (14.1)

N j=1

where the training set consists of the N examples described by the parameter vectors p j ;

the covariance matrix of the components of the parameter vectors:

The existing correlations between the components of the vectors p can be removed by principal component analysis, providing the matrix Pp constructed from the eigenvectors of the covariance matrix . Experience shows that even highly complex organs can well be characterized by the first few eigenvectors with the largest eigenvalues. This results in a description called active shape model [32], which allows to reasonably approximate the full variability of the anatomy by the deviation from the mean shape as a linear combination of a few eigenmodes of variation. The coefficients of this linear combination provide a very compact characterization of the possible organ shapes.

The automatic extraction of the outline of the corpus callosum on midsagittal MR images [33] nicely illustrates the basic ideas of using active shape models for segmentation. Figure 14.4 shows the region of interest covering the corpus callosum on a brain section (a) and on an MR image slice (b). Several examples have been hand-segmented, providing a training set of 71 outlines, which have been parameterized by Fourier coefficients up to degree 100. In order to incorporate not only shape-related but also positional variations into the statistical model, the contours have been normalized relative to a generally accepted neuroanatomical coordinate system, defined by the anterior and posterior commissures (Fig. 14.4). The training data used and the shape model resulting from the principle component analysis is illustrated in Fig. 14.5. As image (b) illustrates,

Figure 14.4: (a) The corpus callosum from an anatomical atlas and (b) the corresponding region of interest in a midsagittal MR image. On the left image the connecting line between the anterior commissure (AC) and the posterior commissure (PC), which is used for normalization, is also shown.

the largest 12 eigenvalues (defined by the 400 original parameters) already reasonably represent the variability (covering about 95% of the full variance).

This statistical description can easily be used as a parametric deformable model allowing the fully automatic segmentation of previously unseen images (apart from the definition of the stereotactic reference system). Based on the

Figure 14.5: Building the active shape model for the corpus callosum. (a) The

71 outlines of the training set normalized in the anatomical coordinate system

defined by the anterior and posterior commissures (AC/PC). The eigenvalues

resulting from the principal component analysis are plotted in (b), while the

eigenvectors corresponding to the three largest eigenvalues are illustrated in

range − 2λk (light gray) to + 2λk (dark gray).

Figure 14.6: Segmentation of the corpus callosum. The top-left image shows the initialization, resulting from the average model and a subsequent match in the subspace of the largest four deformation modes. The other images (top right, lower left, and lower right) illustrate the deformation of this model during optimization using all selected deformation modes, allowing fine adjustments. The black contour is the result of a manual expert segmentation.

concept of deformable contour models or snakes [34] (see section 14.4.3), the corpus callosum outline can be searched in the subspace spanned by the selected number of largest eigenmodes using the usual energy minimization scheme as illustrated in Fig. 14.6. The efficiency of the fit can be vastly increased by incorporating information about the actual appearance of the organ on the radiological image, for example, in the form of intensity profiles along its boundary, as illustrated in Fig. 14.7(a), leading ultimately to the usage of integrated active appearance models [35] incorporating the shape and gray-level appearance of the anatomy in a coherent manner.

The illustrated ideas generalize conceptually very well to three dimensions, as illustrated on the anatomical model of the basal ganglia of the human brain shown in Fig. 14.8. The corresponding active shape model has been successfully applied for the segmentation of neuroradiological MR volumes [36]. Remaining interactions needed for the establishment of the anatomical coordinate system