Handbook_of_statistical_analysis_using_SAS

Chapter 13

Principal Components Analysis and Factor Analysis: The Olympic Decathlon and Statements about Pain

13.1Description of Data

This chapter concerns two data sets: the first, given in Display 13.1 (SDS, Table 357), involves the results for the men’s decathlon in the 1988 Olympics, and the second, shown in Display 13.2, arises from a study concerned with the development of a standardized scale to measure beliefs about controlling pain (Skevington [1990]). Here, a sample of 123 people suffering from extreme pain were asked to rate nine statements about pain on a scale of 1 to 6, ranging from disagreement to agreement. It is the correlations between these statements that appear in Display 13.2 (SDS, Table 492). The nine statements used were as follows:

1.Whether or not I am in pain in the future depends on the skills of the doctors.

©2002 CRC Press LLC

Display 13.1

2.Whenever I am in pain, it is usually because of something I have done or not done.

3.Whether or not I am in pain depends on what the doctors do for me.

4.I cannot get any help for my pain unless I go to seek medical advice.

©2002 CRC Press LLC

5.When I am in pain, I know that it is because I have not been taking proper exercise or eating the right food.

6.People’s pain results from their own carelessness.

7.I am directly responsible for my pain.

8.Relief from pain is chiefly controlled by the doctors.

9.People who are never in pain are just plain lucky.

For the decathlon data, we will investigate ways of displaying the data graphically, and, in addition, see how a statistical approach to assigning an overall score agrees with the score shown in Display 13.1, which is calculated using a series of standard conversion tables for each event.

For the pain data, the primary question of interest is: what is the underlying structure of the pain statements?

Display 13.2

13.2Principal Components and Factor Analyses

Two methods of analysis are the subject of this chapter: principal components analysis and factor analysis. In very general terms, both can be seen as approaches to summarising and uncovering any patterns in a set of multivariate data. The details behind each method are, however, quite different.

13.2.1 Principal Components Analysis

Principal components analysis is amongst the oldest and most widely used multivariate technique. Originally introduced by Pearson (1901) and independently by Hotelling (1933), the basic idea of the method is to describe the variation in a set of multivariate data in terms of a set of new,

©2002 CRC Press LLC

uncorrelated variables, each of which is defined to be a particular linear combination of the original variables. In other words, principal components analysis is a transformation from the observed variables, x1, … xp, to variables, y1, …, yp, where:

The coefficients defining each new variable are chosen so that the following conditions hold:

The y variables (the principal components) are arranged in decreasing order of variance accounted for so that, for example, the first principal component accounts for as much as possible of the variation in the original data.

The y variables are uncorrelated with one another.

The coefficients are found as the eigenvectors of the observed covariance matrix, S, although when the original variables are on very different scales it is wiser to extract them from the observed correlation matrix, R, instead. The variances of the new variables are given by the eigenvectors of S or R.

The usual objective of this type of analysis is to assess whether the first few components account for a large proportion of the variation in the data, in which case they can be used to provide a convenient summary of the data for later analysis. Choosing the number of components adequate for summarising a set of multivariate data is generally based on one or another of a number of relative ad hoc procedures:

Retain just enough components to explain some specified large percentages of the total variation of the original variables. Values between 70 and 90% are usually suggested, although smaller values might be appropriate as the number of variables, p, or number of subjects, n, increases.

Exclude those principal components whose eigenvalues are less than the average. When the components are extracted from the observed correlation matrix, this implies excluding components with eigenvalues less than 1.

Plot the eigenvalues as a scree diagram and look for a clear “elbow” in the curve.

©2002 CRC Press LLC

Principal component scores for an individual i with vector of variable values xi′ can be obtained from the equations:

(Full details of principal components analysis are given in Everitt and Dunn [2001].)

13.2.2 Factor Analysis

Factor analysis is concerned with whether the covariances or correlations between a set of observed variables can be “explained” in terms of a smaller number of unobservable latent variables or common factors. Explanation here means that the correlation between each pair of measured (manifest) variables arises because of their mutual association with the common factors. Consequently, the partial correlations between any pair of observed variables, given the values of the common factors, should be approximately zero.

The formal model linking manifest and latent variables is essentially that of multiple regression (see Chapter 3). In detail,

where f1, f2, …, fk are the latent variables (common factors) and k < p. These equations can be written more concisely as:

where

©2002 CRC Press LLC

The residual terms u1, …, up, (also known as specific variates), are assumed uncorrelated with each other and with the common factors. The elements of Λ are usually referred to in this context as factor loadings.

Because the factors are unobserved, we can fix their location and scale arbitrarily. Thus, we assume they are in standardized form with mean zero and standard deviation one. (We also assume they are uncorrelated, although this is not an essential requirement.)

With these assumptions, the model in Eq. (13.4) implies that the population covariance matrix of the observed variables,ΣΣ , has the form:

whereΨ Ψ is a diagonal matrix containing the variances of the residual

terms, ψ i = 1 … p.

The parameters in the factor analysis model can be estimated in a number of ways, including maximum likelihood, which also leads to a test for number of factors. The initial solution can be “rotated” as an aid to interpretation, as described fully in Everitt and Dunn (2001). (Principal components can also be rotated but then the defining maximal proportion of variance property is lost.)

13.2.3 Factor Analysis and Principal Components Compared

Factor analysis, like principal components analysis, is an attempt to explain a set of data in terms of a smaller number of dimensions than one starts with, but the procedures used to achieve this goal are essentially quite different in the two methods. Factor analysis, unlike principal components analysis, begins with a hypothesis about the covariance (or correlational) structure of the variables. Formally, this hypothesis is that a covariance matrixΣΣ , of order and rank p, can be partitioned into two matricesΛ ΛΛ Λ ′ andΨ Ψ . The first is of order p but rank k (the number of common factors), whose off-diagonal elements are equal to those of ΣΣ . The second is a diagonal matrix of full rank p, whose elements when added to the diagonal elements ofΛ ΛΛ Λ ′ give the diagonal elements ofΣΣ . That is, the hypothesis is that a set of k latent variables exists (k < p), and these are adequate to account for the interrelationships of the variables although not for their full variances. Principal components analysis, however, is merely a transformation of the data and no assumptions are made about the form of the covariance matrix from which the data arise. This type of analysis has no part corresponding to the specific variates of factor analysis. Consequently, if the factor model holds but the variances of the specific variables are small, we would expect both forms of analysis to give similar results. If, however, the specific variances are large, they will be absorbed into

©2002 CRC Press LLC

all the principal components, both retained and rejected; whereas factor analysis makes special provision for them. It should be remembered that both forms of analysis are similar in one important respect: namely, that they are both pointless if the observed variables are almost uncorrelated

— factor analysis because it has nothing to explain and principal components analysis because it would simply lead to components that are similar to the original variables.

13.3Analysis Using SAS

13.3.1 Olympic Decathlon

The file olympic.dat (SDS, p. 357) contains the event results and overall scores shown in Display 13.1, but not the athletes' names. The data are tab separated and may be read with the following data step.

data decathlon;

infile 'n:\handbook2\datasets\olympic.dat' expandtabs; input run100 Ljump shot Hjump run400 hurdle discus polevlt javelin run1500 score;

run;

Before undertaking a principal components analysis of the data, it is advisable to check them in some way for outliers. Here, we examine the distribution of the total score assigned to each competitor with proc univariate.

proc univariate data=decathlon plots; var score;

run;

Details of the proc univariate were given in Chapter 2. The output of proc univariate is given in Display 13.3.

The UNIVARIATE Procedure

Variable: score

Moments

©2002 CRC Press LLC

©2002 CRC Press LLC

The UNIVARIATE Procedure

Variable: score

Display 13.3

©2002 CRC Press LLC

The athlete Kunwar with the lowest score is very clearly an outlier and will now be removed from the data set before further analysis. And it will help in interpreting results if all events are “scored” in the same direction; thus, we take negative values for the four running events. In this way, all ten events are such that small values represent a poor performance and large values the reverse.

data decathlon; set decathlon; if score > 6000;

run100=run100*-1; run400=run400*-1; hurdle=hurdle*-1; run1500=run1500*-1;

run;

A principal components can now be applied using proc princomp:

proc princomp data=decathlon out=pcout; var run100--run1500;

run;

The out= option on the proc statement names a data set that will contain the principal component scores plus all the original variables. The analysis is applied to the correlation matrix by default.

The output is shown as Display 13.4. Notice first that the components as given are scaled so that the sums of squares of their elements are equal to 1. To rescale them so that they represent correlations between variables and components, they would need to be multiplied by the square root of the corresponding eigenvalue. The coefficients defining the first component are all positive and it is clearly a measure of overall performance (see later). This component has variance 3.42 and accounts for 34% of the total variation in the data. The second component contrasts performance on the “power” events such as shot and discus with the only really “stamina” event, the 1500-m run. The second component has variance 2.61; so between them, the first two components account for 60% of the total variance.

Only the first two components have eigenvalues greater than one, suggesting that the first two principal component scores for each athlete provide an adequate and parsimonious description of the data.

©2002 CRC Press LLC