Handbook_of_statistical_analysis_using_SAS

recorded, and which of these five measurements are most informative in this classification exercise.

©2002 CRC Press LLC

Note: X1 = greatest length of skull; X2 = greatest horizontal breadth of skull; X3 = height of skull; X4 = upper face height; and X5 = face breadth, between outermost points of cheek bones.

Display 15.1

15.2Discriminant Function Analysis

Discriminant analysis is concerned with deriving helpful rules for allocating observations to one or another of a set of a priori defined classes in some optimal way, using the information provided by a series of measurements made of each sample member. The technique is used in situations in which the investigator has one set of observations, the training sample, for which group membership is known with certainty a priori, and a second set, the test sample, consisting of the observations for which group membership is unknown and which we require to allocate to one of the known groups with as few misclassifications as possible.

An initial question that might be asked is: since the members of the training sample can be classified with certainty, why not apply the procedure used in their classification to the test sample? Reasons are not difficult to find. In medicine, for example, it might be possible to diagnose a particular condition with certainty only as a result of a post-mortem examination. Clearly, for patients still alive and in need of treatment, a different diagnostic procedure would be useful!

Several methods for discriminant analysis are available, but here we concentrate on the one proposed by Fisher (1936) as a method for classifying an observation into one of two possible groups using measurements x1, x2, …, xp. Fisher’s approach to the problem was to seek a linear function z of the variables:

©2002 CRC Press LLC

such that the ratio of the between-groups variance of z to its within-group variance is maximized. This implies that the coefficients a′ = [a1, …, ap ] have to be chosen so that V, given by:

is maximized. In Eq. (15.2), S is the pooled within-groups covariance matrix; that is

where S1 and S2 are the covariance matrices of the two groups, and n1 and n2 the group sample sizes. The matrix B in Eq. (15.2) is the covariance matrix of the group means.

The vector a that maximizes V is given by the solution of the equation:

In the two-group situation, the single solution can be shown to be:

observations in each group.

The assumptions under which Fisher’s method is optimal are

The data in both groups have a multivariate normal distribution.

The covariance matrices of each group are the same.

If the covariance matrices are not the same, but the data are multivariate normal, a quadratic discriminant function may be required. If the data are not multivariate normal, an alternative such as logistic discrimination (Everitt and Dunn [2001]) may be more useful, although Fisher’s method is known to be relatively robust against departures from normality (Hand [1981]).

means in each group, the classification rule for an observation with discriminant score zi is:

©2002 CRC Press LLC

Assign to group 1 if zi – zc < 0,

Assign to group 2 if zi – zc ≥ 0,

where

(This rule assumes that the prior probabilities of belonging to each group are the same.) Subsets of variables most useful for discrimination can be identified using procedures similar to the stepwise methods described in Chapter 4.

A question of some importance about a discriminant function is: how well does it perform? One possible method of evaluating performance would be to apply the derived classification rule to the training set data and calculate the misclassification rate; this is known as the resubstitution estimate. However, estimating misclassifications rates in this way, although simple, is known in general to be optimistic (in some cases wildly so). Better estimates of misclassification rates in discriminant analysis can be defined in a variety of ways (see Hand [1997]). One method that is commonly used is the so-called leaving one out method, in which the discriminant function is first derived from only n–1 sample members, and then used to classify the observation not included. The procedure is repeated n times, each time omitting a different observation.

15.3Analysis Using SAS

The data from Display 15.1 can be read in as follows:

data skulls;

infile 'n:\handbook2\datasets\tibetan.dat' expandtabs; input length width height faceheight facewidth;

if _n_ < 18 then type='A'; else type='B';

run;

A parametric discriminant analysis can be specified as follows:

proc discrim data=skulls pool=test simple manova wcov cross validate;

class type;

var length--facewidth; run;

©2002 CRC Press LLC

The option pool=test provides a test of the equality of the within-group covariance matrices. If the test is significant beyond a level specified by slpool, then a quadratic rather than a linear discriminant function is derived. The default value of slpool is 0.1,

The manova option provides a test of the equality of the mean vectors of the two groups. Clearly, if there is no difference, a discriminant analysis is mostly a waste of time.

The simple option provides useful summary statistics, both overall and within groups; wcov gives the within-group covariance matrices, the crossvalidate option is discussed later in the chapter; the class statement names the variable that defines the groups; and the var statement names the variables to be used to form the discriminant function.

The output is shown in Display 15.2. The results for the test of the equality of the within-group covariance matrices are shown in Display 15.2. The chi-squared test of the equality of the two covariance matrices is not significant at the 0.1 level and thus a linear discriminant function will be derived. The results of the multivariate analysis of variance are also shown in Display 15.2. Because there are only two groups here, all four test criteria lead to the same F-value, which is significant well beyond the 5% level.

The results defining the discriminant function are given in Display 15.2. The two sets of coefficients given need to be subtracted to give the discriminant function in the form described in the previous chapter section.

Thus, for example, a skull having a vector of measurements x′ = [185, 142, 130, 72, 133] has a discriminant score of –30.07, and z1 – zc in this case is therefore 0.39 and the skull should be assigned to group 1.

©2002 CRC Press LLC

The DISCRIM Procedure

Within-Class Covariance Matrices

--------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------------------------

The DISCRIM Procedure

Simple Statistics

Total-Sample

--------------------------------------------------------------------------------------------------

©2002 CRC Press LLC

--------------------------------------------------------------------------------------------------

type = B

--------------------------------------------------------------------------------------------------

Within Covariance Matrix Information

©2002 CRC Press LLC

is distributed approximately as Chi-Square(DF).

Since the Chi-Square value is not significant at the 0.1 level, a pooled covariance matrix will be used in the discriminant function.

Reference: Morrison, D.F. (1976) Multivariate Statistical Methods p252.

The DISCRIM Procedure

Pairwise Generalized Squared Distances Between Groups

Generalized Squared Distance to type

©2002 CRC Press LLC

The DISCRIM Procedure

Classification Summary for Calibration Data: WORK.SKULLS Resubstitution Summary using Linear Discriminant Function

Generalized Squared Distance Function

Posterior Probability of Membership in Each type

2 2

©2002 CRC Press LLC

Number of Observations and Percent Classified into type

Error Count Estimates for type

The DISCRIM Procedure

Classification Summary for Calibration Data: WORK.SKULLS Cross-validation Summary using Linear Discriminant Function

Generalized Squared Distance Function

Posterior Probability of Membership in Each type

2 2 Pr(j|X) = exp(-.5 D (X)) / SUM exp(-.5 D (X))

j k k

©2002 CRC Press LLC