Kluwer - Handbook of Biomedical Image Analysis Vol

of the 20 bee brains was taken as the raw image and automatically segmented using every one of the remaining 19 brains as an atlas. This resulted in 19 segmentations per brain. These 19 segmentations were then combined into a final segmentation.

The most straightforward method for combining multiple classifications into one is the so-called “Vote Rule” decision fusion [26]. For each voxel in the raw image, the outputs of the individual atlas-based classifiers are determined. Their “votes” are then counted, and the label that received that highest number of votes is assigned to the voxel. It is worth, however, to take a closer look at the way an atlas-based classifier works: by looking up a label according to a transformed image coordinate. The label map is discrete, arranged on a 3D grid of labeled voxels. Yet the coordinates of the raw image voxels that we are trying to label hardly ever directly fall on grid points in the atlas. Therefore, looking up the correct label requires some sort of interpolation. The simplest label interpolation method is nearest neighbor (NN) interpolation, resulting in a single unique label per atlas-based classifier. These can easily be combined using vote fusion as described above.

A slightly more complex interpolation technique that can be applied to labels is partial volume interpolation (PVI) as introduced by Maes et al. [36]. Here, the labels of all eight neighbors of the interpolated coordinate are determined and weighted with the trilinear interpolation coefficients of their respective grid nodes. Therefore, the output of an atlas-based classifier using PVI is a vector of weights between zero and one, which are assigned to each of the possible labels. One can interpret the weights as the confidence of the classifier in the respective label being the correct answer. These weighted decisions from all classifiers can be combined by so-called “Sum Rule” fusion [26]. The weights for each label are added over all classifiers, and the label with the highest sum is taken as the combined decision.

11.5 Quantifying Segmentation Accuracy

In addition to presenting selected algorithms for atlas-based segmentation, this chapter provides a quantitative comparison among different methods. For each segmentation that we perform, its accuracy is computed. The accuracies

achieved for each image are then compared among different methods in order to illustrate quality differences and to identify superior algorithms.

Computing the accuracy of a segmentation requires a gold standard, or ground truth. That is, the correct segmentation needs to be known for an image in order to be able to compute the accuracy of an automatically generated segmentation of that image. While not at all guaranteed to be correct, it is commonly accepted today to use a manual segmentation by a human expert, supported by advanced semi-automatic labeling techniques such as intelligent scissors [40], as the gold standard that automatic segmentation methods are measured against.

11.5.1 Similarity Index

Figure 11.11 provides a visual impression of the segmentation result for two representative slices from one segmented bee brain image. However, in order

Figure 11.11: Example of segmentation using non-rigid image registration (MUL atlas selection paradigm). The two columns show axial images at two different slice locations. Top row: Overlays of segmentation contours (shown in white) after non-rigid image registration. Bottom row: Difference images between manual and automatic segmentation. Voxels with different labels assigned by manual and automatic segmentation are shown in black.

Figure 11.12: Dependence of SI values on size for spherical objects. The squares show SI values computed from discrete numerical simulation of dilation by one voxel. The solid line shows SI values for the continuous case (Eq. 11.9). Note that while the units on the horizontal axis are voxels for the discrete case, they are arbitrary units for the continuous case.

et al. [16] reported mean SI values of 0.96 for segmentation of the human brain from MR images and mean SI values of only 0.85 for segmentation of smaller brain structures such as the caudate.

In Fig. 11.13, the average volumes of the anatomical structures in the bee brain images under consideration are shown with the actual segmentation accuracies achieved for them using one of the segmentation methods discussed later (MUL). It is easy to see that the larger a structure, the more accurately it was typically segmented by the atlas-based segmentation. This confirms the theoretical treatment above and illustrates the varying bias of the SI metric when segmenting structures of different sizes.

11.5.3 Bias from Structure Shape

A simple numerical measure that characterizes the shape of a geometrical object is its surface-to-volume ratio (SVR). For a discrete set of labeled voxels in a segmented structure, we can approximate the SVR ρ as the ratio of the number of surface voxels Ns to the total number of voxels Nt , that is,

Nt

Figure 11.16: SI by label for segmentation using a single individual atlas (IND atlas selection strategy) [reproduced from [53]].

the pattern recognition community that multiple classifier systems are generally superior to single classifiers [27, 83].

Another interesting finding is that both the AVG and the MUL strategies performed better than the theoretical upper bound of any strategy working with only a single individual atlas (series labeled “Best SI” in Fig. 11.20). We note that “Best SI” is the upper bound not only for any method with the best atlas for each raw image, but also for any possible selection of one atlas for all raw images. Therefore, it is also the upper bound of the IND strategies, which in our study can consequently never outperform the AVG or MUL strategies.

Figure 11.17: SI by label for segmentation using the most similar single individual atlas (SIM atlas selection strategy) [reproduced from [53]].

Figure 11.20: Percentage of registration-based segmentations with similarity index SI better than the given threshold plotted by atlas selection strategy. The series labeled “Best SI” is the upper bound of all strategies working with a single individual atlas (see text for details).

strategy was that different atlases lead to segmentations of different accuracies. Combined, both observations lead to an even more interesting concept: combination of multiple atlas-based segmentations, weighted by estimates of their individual segmentation accuracy.

In other words, if we had estimates of how well each atlas-based classifier is performing, then we could be more confident in decisions of those classifiers that perform well, compared to the decisions of those that do not. One would hope that by concentrating on more accurate classifiers in the ensemble, the classification accuracy would be further improved.

The performance of each atlas-based classifier is obviously not known in general, due to the lack of a ground truth. However, several methods have been proposed that can estimate the performance parameters, for example, using expectation maximization (EM) methods. Two of these are outlined below, one based on a per-label binary performance model [79], and another based on a simultaneous multilabel performance model [60, 61].

For the description of both methods, we assume that an image with N voxels is segmented by K different (atlas-based) classifiers. For each voxel x, we denote with ek(x) the decision by classifier k, which is one of the labels assigned in