Словари и журналы / Психологические журналы / p1British Journal of Mathematical and Statistical Psychology

2 David J. Bartholomew and Shing On Leung

we require a means of testing whether the Žt is adequate. The obvious way is to compare the observed and expected frequencies for each response pattern by the usual chi-squared or log-likelihood ratio tests, but in practice this is often difŽcult. The reason is that if 2p is large compared with N, the sample size, many expected frequencies will be too small for the x2 approximation to the sampling distribution to be valid. For example, in a typical case with p = 10 and N = 200 there are 210 = 1024 possible response patterns (cells). On average, the expected frequency will be 0.2 and there will therefore be many much less than this. The difŽculty may be partially overcome by grouping response patterns, but this often has to be so drastic as to make the number of degrees of freedom zero or negative.

There have been several studies of this problem as it arises in latent variable and loglinear modelling (for example, Collins, Fiddler, Wugalter, & Long, 1993; Reiser & VandenBerg, 1994; Langeheine, Pannekoek, & van de Pol, 1996; Reiser, 1996; van der Heijden, Hart, &Dessens, 1997; Bartholomew &Tzamourani, 1999; Reiser & Lin, 1999). These authors generally agree that the usual test may be invalid and propose various ways of dealing with the problem. Bartholomew and Tzamourani (1999), for example, propose a bootstrap approximation supplemented by various diagnostic checks based on residuals. The approach which comes closest to our own is that followed by Reiser (1996) and continued in Reiser and Lin (1999). They propose tests based on Žrstand second-order margins and give a review of the reasons for doing so. We shall compare our approach with their’s in Section 5.

2. The test statistic

In this paper we propose a new, global, test based on the two-way margins of the contingency table. The two-way margins give the frequencies of responses for each of the 12 p(p ± 1) pairs of variables. Thus for variables i and j they will give the frequencies of the responses (1, 1), (1, 0), (0, 1) and (0, 0). If one of these frequencies is known the others are determined from the one-way margins which are regarded as Žxed since they provide no information about pairwise associations. Let nij be the observed frequency for the response (1, 1) on variables i and j. Then our test statistic is deŽned as

where Npij is the expected value of nij. The (i, j)th term in Y is a measure of the departure of the bivariate distribution of that pair of variables from that predicted by the model. Since nij is a binomial random variable with parameters N and pij, the individual terms in (1) are distributed approximately as x2 with one degree of freedom. However, the terms are not independent, because the nij are not independent, and so it is not true that Y has a x2 distribution with 12 p(p ± 1) degrees of freedom. In the next section we

show that the distribution of Y can be closelyapproximated by that of a linear function of x2.

As we have deŽned it, the test is for the case where the pij —and hence the model parameters—are known. In practice, this is rarely the case and we shall then have to replace them by sample estimates. We shall investigate the effect that this further approximation has on the distribution.

3. The approximation to the sampling distribution of Y

Preliminary simulation studies of the sampling distribution of Y for latent trait models showed that its form is similar to the central x2 distribution, especiallyin the upper tail. Since it is possible to derive the exact moments of Y, this suggested Žnding a linear function of x2 which has the same Žrst few moments as Y. This is the basis of our approximation. In the next two subsections we show how to obtain the exact and asymptotic moments of Y: the latter provide a useful approximation to the former.

3.1. The exact moments of Y

Since E(nij ± Npij )2 = Var(nij) = Npij(1 ± pij), it follows that E(Y) = 12 p(p ± 1) whatever the model. The higher moments depend on the moments and cross-moments of the

nij. The derivation is straightforward but tedious, and proceeds as follows.

Since only the nij are random variables, the denominator is a constant. E(Y2) will

depend on the expectation of n2ijn2kl, n2ijnkl, nij n2kl, nijnkl , n2ij , n2kl, nij and nkl. Further details are set out in the Appendix.

The third moment can be obtained by the same method, but the algebra is much heavier. Because of its complexity we obtained numerical values using a Fortran program which Žrst calculates the raw moments and cross-moments of the ns and then substitutes them into the expression for E(Y3). As a partial check on the veryheavy algebra, we also estimated the moments from 10 000 random samples from each of three models which were generated for checking the approximations in Section 3.2 below. Some results are given in Table 1. The entries for N=100, 500 and 1000 in the columns headed ‘Exact’ were calculated from the formulae in the Appendix for the second moment and using the algorithm for the third moment referred to above; the values for N = ¥ were derived by the method given in the next section. The three models are: an ‘independence model’ which supposes the Xs are mutuallyindependent; and two latent trait models (see Section 4.1 below) which are typical of cases found in practice.

The parameter values used are as follows. In the case of the oneand two-factor

a1 = (0.4, 0.8, 1.2, 1.6, 2.0, 1.0, 1.0, 1.0, 1.0, 1.0)¢ and a2 = (1.0, 1.0, 1.0, 1.0, 1.0, 0.4, 0.8, 1.2, 1.6, 2.0)¢. For the independence model the ai1 are zero and the ai0 were chosen so

that the Žrst-order margins are the same as those for the one-factor model. These values are typical of those encountered in practice.

The agreement between the exact and simulated moments was generally good, and where there were discrepancies they were not systematic.

3.2. The asymptotic moments of Y

In order to obtain an approximation which can be implemented without the cumbersome calculations required to obtain the exact moments, we now show that asymptotic values

can be obtained relatively easily. This method may also be extended to obtain higher moments of Y and moments for other quantities of a similar form not discussed here.

We may write

S may be written AL A¢, where L is a diagonal matrix of eigenvalues and A is the

= 2tr(S2).

Similarly, the asymptotic third moment of Y is the sum of the cubes of the ls multiplied by the third moment of a x2 variable with one degree of freedom, which is 8tr(S3). The element of S in the row labelled (i, j) and the column (k, l) is easily shown to be

p •

(pijkl ± pijpkl)/ DijDkl, where Dij = pij(1 ± pij ). The probabilities appearing in this expression are fourthand second-order marginal probabilities respectively and can

be calculated from the expected cell frequencies which are part of the standard output of the programs for Žtting latent variable and other models. Hence the asymptotic moments are easilyobtained by raising S to the appropriate power or by computing the eigenvalues. Numerical values are given in Table 1, which shows that the exact moments rapidly approach their limits.

3.3. The chi-squared approximation

We have alreadynoted that the simulated distributions appear to be similar in form to x2 in the upper tail so that it is reasonable to approximate the distribution of Y by treating it as a linear function of a x2 random variable. Thus we assume that

As we have expressions for the Žrst three moments of Y we can choose a, b and c so that the approximating distribution has the same Žrst three moments as Y.

Equating moments, it is easily shown that

b = m3(Y) , 4m2(Y)

c = m2(Y) ,

2b2

a = m1(Y) ± bc,

where m1(Y), m2(Y) and m3(Y) are the central moments of Y.

This we shall call the three-moment approximation. The range of the approximating distribution is (a, ¥) rather than the true range of (0, ¥). In practice this is of no importance because, in testing goodness of Žt, we are only interested in the upper tail. However, a simpler approximation, with the correct range, is obtained by setting a = 0. If we now determine b and c so that the Žrst two moments agree, we have

This is the two-moment approximation. Each approximationcan be made using either the exact moments or the asymptotic moments. Thus, altogether we have four approximations which are compared in Table 2. We have done this byestimating the 10%, 5%and 1% points of the exact distributions from the simulated distributions based on 10 000 replications. The values of a, b and c were calculated once for each cell of the table using the exact moments. We then estimated the probability for each approximation using (6). The three models are the same as those used for the calculations in Table 1.

All the approximations would be adequate for most practical purposes. The threemoment approximation is usuallythe best, especiallyat the smaller P-values, but for the one-and two-factor models there are cases where the two-moment approximations are slightly better. The difference between the exact and asymptotic moments is negligible and we therefore recommend the approximation based on asymptotic moments which will be used in the examples that follow. It should be borne in mind that the exact P- values are subject to some uncertainty, having been computed from 10 000 simulations.

As we noted in Section 2, the approximation has been derived and tested on the assumption that the model parameters are known. When the parameters are replaced by estimators there will be an effect on both the statistic and the sampling distribution. Asymptotically this will be negligible because the substitution introduces an error of order N± 1/2. However, it would be useful to know how large the sample size needs to be before the approximation can be ignored. Afull investigation of this point will require a major simulation study, but a preliminary calculation has been made for the case N = 100 and p = 10 using the same parameter values as in the simulation of Section 3.1. The sampling distribution of the estimated Y was simulated using 1067 replications. The percentages exceeding the 10%, 5%and 1%points of Y (as estimated from 10 000 simulations) were 9.37%, 4.87%and 0.84%, respectively. This suggests that, even with a sample size as small as 100, the procedure we have proposed will be adequate.

Areferee has pointed out that using the distribution of Y as an approximation to that of Yˆ is likelyto be inadequate if the model Žtted is poorly identiŽed. To take an extreme case, if the model is just identiŽed, the observed and expected frequencies will be equal and Yˆ will therefore be zero and its distribution will be concentrated at a single point. The distribution of Y, on the other hand, will not be so constrained. This is true but will not be relevant for the problems for which our test is designed. Yˆ is speciŽcally intended for sparse tables, and sparseness is only likely to arise when 2p is of at least the same order as the sample size. In such cases it will also be far in excess of the number of parameters to be estimated. This is why we chose p = 10 for the preliminary calculations given above; for models like those Žtted here 210 greatly exceeds the number of parameters.

4. Applications

4.1. Sexual attitudes data

We Žrst illustrate the test by carrying further the analysis of an example on sexual attitudes given in Bartholomew and Knott (1999, Example 2, p. 143). There a latent class model was Žtted. Here we Žt a latent trait model. That is, we investigate whether the associations among the responses can be explained by a common dependence on one or more latent variables. Let

where z is a q-vector of latent variables assumed, without loss of generality, to have zero means and unit variances. The logit/normit model then speciŽes that

logit pi(z) = ai0 + ai1z1 + ai2z2 + . . . , i = 1, 2, . . . , p, (7)

where the zs are mutually independent and normally distributed. We investigate the Žt of this model for q = 1 and 2 using the methods and software available in Bartholomew and Knott (1999). Some results are given in Table 3.

The table gives that part of the analysis needed to apply the Y test. There are p questions and hence 12 10(10 ± 1) = 45 two-way margins as listed in the Žrst column. The columns headed ‘Contribution’ are the individual terms in (1) and the column totals are, therefore, the values of Y for the two models. Before carrying out the formal test it is immediately clear that the two-factor model is an excellent Žt, with a value of Y well below its expectation of 45. The value of Y for the one-factor model is larger, and almost half its value arises from the last row where the contribution is 49.37.

Using the x2 approximation we obtain the results given in Table 4 for the asymptotic threeand two-moment approximations. We have given only the moments and not the intermediate calculations required to obtain them. These have been deferred to Section 4.2 where the smaller value of p (= 5) allows a more economical presentation of the details.

With a p-value of 0.065 the model with p = 10 is only marginally signiŽcant but, having regard to the the extremely large contribution for the (9, 10) margins we consider that the two-factor model is more appropriate. This is consistent with the results obtained in de Menezes and Bartholomew (1996) for the same data. These indicated that the two questions on adoption by homosexual couples raise a quite different issue than the remaining items. A one-factor Žt would suggest a single dimension of variability according to a traditional/permissive scale of attitude to sexual behaviour. The present analysis conŽrms that this is not sufŽcient to explain attitudes to adoption. If the one-factor model is Žtted to items 1–8, excluding the adoption questions, the right-hand part of the table shows that the agreement is good, as we would expect.

4.2. Social life feelings and the workplace industrial relations survey

Here we return to two examples analysed in Bartholomew (1998), where further details will be found. The sparsity problem was not as great in these cases; for the social life feelings (SLF) data, N = 1490 and p = 5; for the workplace industrial relations survey (WIRS) data, N = 1005 and p = 6. Nevertheless, there were a number of very small expected frequencies and some uncertainty about the interpretation of the tests. The new test enables us to throw some new light on these unresolved questions.

For the SLF data the usual goodness of Žt tests (x2 and G2) gave values of 32.87 and