Словари и журналы / Психологические журналы / p41British Journal of Mathematical and Statistical Psycholog

42 Ke-Hai Yuan and Wai Chan

Bollen, 1989). Because of the complexity of real-world data, various methods for SEM have been developed that do not require the normality assumption (e.g., Browne, 1984; Bentler & Dijkstra, 1985; Satorra, 1992; Satorra & Bentler, 1988; Kano, Berkane, & Bentler, 1993; Yuan & Bentler, 2000). These methods lead to better parameter estimation and model evaluation when a data set does not follow the normal distribution assumption. For example, one may use the asymptotically distribution-free (ADF) method (Browne, 1984; Yuan & Bentler, 1997a) for model estimation and evaluation when a data set does not follow a normal distribution. However, there still exist various issues that are difŽcult to deal with in the SEM literature.

One practical issue concerns improper solutions or Heywood cases (negative error variance estimates). In the classical example summarized in Jo¨ reskog (1967), factor models for 9 out of 11 classical data sets possess improper solutions. There are various causes of improper solutions. Van Driel (1978) identiŽed three major ones: sampling •uctuations combined with small true values of error variances; model misspeciŽcation; and indeŽniteness of the model. A Monte Carlo study conducted by Anderson and Gerbing (1984) with correct models found that 24.9% of replications had improper solutions. One approach to improper solutions is to remove the variables corresponding to negative estimates of error variances (Lawley&Maxwell, 1971). This may not be ideal because of the loss of essential information provided by these marginals. In the context of unrestricted factor analysis, Martin and McDonald (1975) proposed a Bayesian approach to overcome Heywood cases. However, it is unclear how to obtain standard errors of parameter estimates and how to test the overall model adequacy with this approach.

Another common issue arises when a multivariate data set consists of marginals which are jointly normal and marginals which are jointly far away from normal. In such a case one may use the ADF method to Žt the model in order to obtain more efŽcient parameter estimators. It is known that parameter estimators based on the ADF method are at least as efŽcient as those based on the normal theory maximum likelihood (ML) when sample sizes are large enough. For small to medium-sized samples, however, empirical studies indicate that the maximum likelihood estimators (MLEs) are more efŽcient, especially when data are approximately normal (Yuan & Bentler, 1997b). In order to obtain more efŽcient parameter estimates one would like to estimate parameters in the submodel corresponding to the nearly normal variables by ML and those corresponding to marginals that are far away from normal by the ADF method. However, it is not clear how to combine the separate estimation procedures in evaluating the overall model structure and in estimating the parameters that describe the relations among the different sets of variables.

The third problem occurs when only a single indicator is available for a latent construct. If the reliability coefŽcient of this indicator is known, then the measurement error variance may be replaced by its estimate and treated as a constant in subsequent analysis. However, as the estimated variance is still random, it is not clear how treating a random number as a constant affects the other parameter estimates and the evaluation of the overall model in the subsequent analysis.

The last issue arises when convergence cannot be reached, and this is probably the most challenging issue facing many SEM users. The main causes of non-convergence are small sample sizes associated with large models and when the model does not Žt data. Even for correct models in the Monte Carlo study a certain proportion of non-convergences still exists (Anderson & Gerbing, 1984; Yuan & Bentler, 1997a). The problem is that, without convergence, one has no way of judging the causes

As we shall see, all the above issues can be tackled by means of the approach of estimating equations with nuisance parameters. Suppose we have a covariance structure S = S(b). Somehow, b contains a subvector which either has already been estimated or needs to be estimated separately. Let this vector be g, and b = (u¢, g¢)’. Now our problem becomes that of how to estimate u in S = S(u, g). Because an estimate gˆ is already available, one faces the problem of minimizing a discrepancy function

in the case of maximum likelihood. Of course, the ADF discrepancy function (Browne, 1984) can also be chosen here. The vector gˆ in (1) may be a consistent estimate of a set of error variances that are improper if solving u and g simultaneously in the conventional approach. It may be a set of interesting model parameter estimates in a separate model estimation due to marginal variables that are far away from normal. With a single indicator, gˆ will be the error variance estimate when a reliability coefŽcient of the indicator is available. In our experience, non-convergence generally occurs with a large model of many free parameters or when the sample size is not large enough relative to the model size. When convergence cannot be reached, one may be able to achieve it by Žtting the segregated smaller models based on different marginals. For example, with a nine-variable, three-factor model, one may Žt three three-variable models for the nine factor loadings and nine error variances. In this case, the gˆ will be these parameter estimates. The parameter vector u will contain only the three factor correlations. Because four smaller models are Žtted it is very likely that one will achieve convergence even when the simultaneous estimation for u and g does not converge.

The above procedure of Žtting S(u, g) with g estimated separately from u is also related to the idea of Žtting measurement and structural models separately as recommended by Anderson and Gerbing (1988). In the same paper, the authors also recommended unidimensionality of measurement. In such a case, g may be the vector of parameters in the measurement model (factor loadings and error variances) obtained in the Žrst stage, and u will be the vector of parameters in the structural model. In the LISREL notation for SEM (Jo¨ reskog & So¨ rbom, 1993), g will be the vector containing the free parameters in Lx, Ly, Qd and Q«; u will contain the free parameters in B, G, F and W. There are many ways to choose g depending on model Žtting convenience. As will be illustrated in the examples, it is up to the researcher in the context of Žtting a speciŽc model to a given data set.

Our aim is to study how the statistical properties of uˆ are related to those of gˆ in this model segregation approach, though we are also interested in how to evaluate the overall model structure. Actually, uˆ = uˆ (gˆ ) is a function of gˆ , and this is characterized through estimating equations with nuisance parameters in Section 2. How the efŽciency of gˆ in•uences that of uˆ in three commonly used discrepancy functions will be studied in Section 3. Section 4 will study the effect on the commonly used test statistics of treating gˆ as known. Several applications will be presented in Section 5. A simulation example in Section 6 contrasts different procedures when the overall model structure is correct. Some concluding remarks will be given in Section 7.

44 Ke-Hai Yuan and Wai Chan

2. Estimating equation with nuisance parameters

Let F[S, S(u, gˆ )] be the discrepancy function which is minimized to give an estimate uˆ . Taking the derivative of F with respect to u, one obtains

where the subscript n is used to indicate that the function Gn depends on the sample size n. The function Gn (S, u, g) is called an estimating function by Godambe (1960). Equation (2) is generallyreferred to as an estimating equation with nuisance parameters (Liang & Zeger, 1995). As mentioned in Section 1, obtaining gˆ by a method other than solving (2) is preferable to solving (u, g) simultaneously. Yuan and Jennrich (2000) studied the distribution of uˆ and its relationship to that of gˆ under quite general conditions. Based on the Žndings of Yuan and Jennrich, this section gives a brief introduction to a result that is relevant to the study of the distribution of uˆ in the next section. We also present some new results regarding the asymptotic covariance matrix of uˆ (gˆ ) in the SEMcontext. We use a dot on top of a function to imply a derivative, and when the function contains more than one argument we use a subscript to denote the corresponding partial derivative; the argument of a function is omitted if evaluated at the population value b0 = (u¢0, g¢0)¢. Let

As we shall see, we need to use part of the data to estimate g0. Actually, gˆ is an implicit

Assuming (3), then Theorem 1 of Yuan and Jennrich (2000), together with their equations (2.3) and (2.4), implies that

It follows immediately from (4) that if B = 0, then the asymptotic distribution of gˆ does p •

not in•uence the asymptotic distribution of uˆ as long as gˆ is n-consistent for g0, so knowing gˆ is equivalent to knowing g0. Yuan and Jennrich (2000) discussed various contexts for B = 0. For the issues highlighted in this paper, the corresponding B will not be zero in general. When B Þ0, the asymptotic distribution of gˆ does in•uence the asymptotic distribution of uˆ . The effect can be seen from the asymptotic covariance

matrix of uˆ . When Gn(S, u, gˆ ) = FÇMLu[S, S(u, gˆ )] and S is the sample covariance matrix based on a normal sample x 1, . . . , x n , then uˆ corresponds to the pseudo-MLE (Gong &

Samaniego, 1981; Parke, 1986; Kano et al., 1993). Parke (1986) observed that for the pseudo-MLE, V12 = 0 generally. Another case where V12 = 0 is when gˆ is asymptotically efŽcient (Pierce, 1982). In the pseudo-MLEcontext, A = ± V11, and the Q matrix can be

Because data in the social and behavioural sciences are typically not normal, other discrepancy functions than (1) are also commonly used in the SEM literature. Even for the ML procedure V12 may not be zero because of the non-normality of the data. It is also difŽcult for one to verifywhether the gˆ s that solve the issues highlighted in Section 1 are efŽcient or not. In the following we will study cases of V12 = 0 under less stringent conditions. We will need to borrow some concepts from the literature on estimating functions (see Godambe, 1960; Godambe & Kale, 1991). Let Gn (S, b) be an estimating function associated with estimating b. For example, Gn (S, b) = FÇMLb(S, b) in the context of using the maximum likelihood discrepancy function. Denote CG = E[GÇnb(S, b)]. A function Gn is called optimal if

C± 1Var(Gn )C¢± 1 ± C± 1Var(Gn )C¢± 1 $0 (6)

G G G G

for all the forms of Gn that satisfy E[Gn(S, b)] = 0. Kale (1985) and Small and McLeish (1988) established a one-to-one relationship between the optimal estimating function Gn and the optimal estimate b˜ by solving

functions in the SEM context may not enjoy Žnite-sample optimal properties, so we will have to redeŽne CG by CG = limn !¥ E[GÇnb(S, b)], Var(Gn ) by the asymptotic covariance matrix of Gn, and E[Gn(S, b)] = 0 by limn !¥ E[Gn (S, b)] = 0 for an asymptotically optimal estimating function.

Let h (b) be a function of b and h¯ (S) be its consistent estimate. A key result here is that if Gn (S, b) is an asymptotically optimal estimating function, then it is asymptotically uncorrelated with h¯ (S). This can be easily proved since, otherwise, we can construct another estimating function

Gn (S, b) = Gn(S, b) + aK[h¯ (S) ± h (b)],

which will violate (6) if a is chosen properly, where K is any given non-singular matrix of proper dimension. A similar technique is used in the proof of Theorem 7.3.3 of Casella and Berger (1990, p. 317).

With b = (u¢, g¢)’ in the SEM context, the estimating function in (2) is generally a subvector of Gn (S, b). When gˆ = h¯ (S) is asymptoticallyuncorrelated with Gn (S, b), gˆ is also asymptotically uncorrelated with Gn (S, u0, g0), so V12 = 0 in such a case. When V12 = 0 in the context of estimating equations,

The second term on the right-hand side of either (5) or (8) re•ects the cost of estimating the extra parameter g0 in (2). The matrix A may not equal the matrix ± V11 in general. For an optimal estimating function Gn there exists A = ± V11, as will be the case for both the pseudo-MLEand the estimating equation derived from the ADFdiscrepancy function considered in the next section.

Notice that the function h¯ (S) of S being asympototically uncorrelated with Gn (S, b) does not exclude some other kind of nonlinear function of the raw sample being correlated with Gn (S, b). For example, a robust estimate gˆ = h¯ (x 1, . . . , x n ) of g may be correlated with Gn (S, u0, g0) when the sample has heavier tails than those of the multivariate normal distribution.

46 Ke-Hai Yuan and Wai Chan

3. Distribution of uˆ for three estimating functions

The theory of estimating equations can be applied to various contexts (Liang & Zeger, 1995; Yuan &Jennrich, 2000). In this section we will use it to solve various issues in SEM as highlighted in Section 1. The estimating equations to be studied include that derived from the normal theory ML discrepancy function (1) when data are normal and nonnormal; that derived from the generalized least squares (GLS) discrepancy function based on normal theory (Browne, 1974), and that from the ADF discrepancy function. For a p ´ p sample covariance matrix S = (sij), let s = vech(S) be the vector formed by stacking the columns of S, omitting the elements above the diagonal. Let vec(S) be the vector of stacking the columns of S, then there exists a unique p2 ´ p matrix Dp such that vec(S) = Dp vech(S) (Magnus & Neudecker, 1999), where p = p( p + 1)/2. We will use s(b) = vech[S(b)]. The estimating functions corresponding to the normal theory MLand GLS discrepancy functions are respectively given by

Let y i = vech[(x i ± x¯ )(x i ± x¯ )¢] and Sy be the sample covariance matrix of y i. The estimating function corresponding to the ADF discrepancy function is

GADF(S, u, gˆ ) = sÇ¢ (u, gˆ )S± 1[s ± s(u, gˆ )]. (9c)

u y

It has been proved by Godambe (1960) that score functions are optimal. Consequently GML is asymptotically optimal in estimating u when data are normal, and so is GGLS

and the ones for the estimating equation corresponding to the ADF discrepancy function are given by

Notice that matrices A and B only involve the covariance structure of S(b) and do not depend on the distribution of the sample or on that of gˆ once a discrepancy function is chosen. On the other hand, the V matrix in (3) depends on the distribution of the data and on that of gˆ . When the data are normal,

for (9c).

For anyof the issues described in Section 1 one has to use part of the data to estimate g. There are various ways of estimating g, depending on the speciŽc context of applications. When a Heywood case occurs in Žtting the overall model simultaneously for uˆ and gˆ , one may Žrst Žt a submodel that contains the parameter with the improper solution. Usually, there is more than one possible choice for this submodel. In this case, the one-factor measurement model with its corresponding indicators, including the problematic item, may serve the purpose. Let x = (x ¢1, x ¢2)¢ represent the observed variables with dimensions p1 and p2 for x 1 and x 2, respectively. When gˆ was obtained based on x 1, we denote the corresponding covariance matrix of x and its corresponding sample counterpart respectively as

Suppose the submodel structure is S11(a) and a contains all the elements of g but mayonly contain a subset of u. Then one can Žt s1(a) = vech[S11(a)] to s1 = vech(S11)

P = H(sÇ¢1W1 sÇ1)± 1sÇ¢1W1L

in the normal theory MLor GLS approach, and

P = H(sÇ¢1W2sÇ1)± 1sÇ¢1W2L

in the ADF approach, with H and L being selection matrices such that g = Ha and

s1 = Ls.

When the data contain marginals that are approximately normal and marginals that are quite non-normal, or when Žtting the whole model simultaneously cannot lead to

With a single indicator x = t + e for the latent construct t, suppose also that the

about its standard error is available, then one may treat r as a constant and we have

of r is available and r ± r0 ,N(0, d2), we can also obtain the distribution of gˆ and the corresponding V matrix in (3). For this we need to assume that d2 is comparable with

where P = ((1 ± r0)L, ± jxx).

With normal data, V12 = 0 for any of the estimating functions in (9). Since the estimating function deŽned in (9c) is asymptotically optimal, in general V12 = 0 for the estimating equation based on the ADF discrepancy function for both normal and non-normal data. When using (12a) to (12c) for non-normal data, it follows from (9) that

for the estimating equations derived from the normal theory based discrepancy functions.

The covariance matrix V22 depends only on the distribution of gˆ for (12a) to (12c),

Because the estimate r in (12d ) depends on previous data, it needs special attention. Notice that, when the reliability estimate r is based on previous research, s and r are independent. When using the normal theory based estimating equations for uˆ , V12 = 0 for normal data; and it follows from (9) and (12d ) that

when the data are not normal. Because the estimating function in (9c) is asymptotically optimal, V12 = 0 for the estimating equation based on the ADF discrepancy function. The variance v22 is given by

and (14) in (4). In particular, the asymptotic covariance matrix of u(gˆ ) can be simpliŽed to (5) or (8) in various cases. In the following, we wish to compare the efŽciency of uˆ in the model segregation approach with that of u˜ in the simultaneous estimation approach. Although, as discussed in Section 1, the simultaneous estimation approach may not be feasible or may lead to an inappropriate solution to g, the comparison will give us more insight to the model estimation processes. This will be studied for the following two cases.

In the Žrst case the data are normal and the normal theory based MLis used in the simultaneous estimation approach. Denote the information matrix in the simultaneous estimation by

Now, regarding F[S, S(bˆ )] as a function of sˆ = vech[S(bˆ )] and using a Taylor expansion of F[S, S(bˆ )] at s gives

F[S, S(bˆ )] = (s ± sˆ )¢W(s ± sˆ ) + op (1/n).

where x21j are independent chi-square variates with one degree of freedom. Unless t1 = . . . = tp ± q, no commonly used distribution is available to describe the behaviour of the right-hand side of (21), which is a linear combination of chi-square distributions. Several approximations to a linear combination of chi-square distributions were developed by Box (1954) and Satterthwaite (1941). Another one was studied by Bentler (1994) for approximating (21) with Sn = S based on a sample from a non-normal distribution. When S is based on a sample from an elliptical distribution, Browne (1984) proposed a rescaling factor for nF(S, S(b˜ )) using Mardia’s (1970) coefŽcient of kurtosis. Satorra and Bentler (1988, 1994) proposed a more general rescaling factor for the likelihood ratio statistic. Tyler (1983) studied the likelihood ratio test for an explicit function of the covariance matrix with a rescaling factor. Replacing the sample