Словари и журналы / Психологические журналы / p63British Journal of Mathematical and Statistical Psycholog

Column 2 shows results from the instrumental variable strategy. This Žrst alternative to the sample selection strategy corrects the variables that have biases with respect to the measured outcomes byusing substitutes. These substitutes are comprised of the predicted scores that arise when using unbiased variables to predict the biased variables. That is, the biases in theory disappear because the predicted scores are based on unbiased variables.

In the present case, the dummy variables that represent the experimental conditions are treated as biased since the clients may accept or reject the experimental conditions based on their perceptions of the outcomes they might obtain. Accordingly, these variables are replaced by the predicted scores from probit equations that explain acceptance of the experimental conditions by variables that are not likely to be affected by the outcomes. These are the same independent variables that predict acceptance into the condition in the sample selection analyses. The instrumental variable strategy eschews the assumption that the data on the refusers are missing at random.

For the present experiment, separate equations are used to predict acceptance of the housing and the case-management conditions. There is some awkwardness in this application because the probit equations must be set up something like the sample selection equations, which means that they have similar estimation problems. Nevertheless, results can be obtained.

Table 7 suggests that, when the instrumental variables are substituted for the interventions in outcome equations, the case-management intervention has large, statistically signiŽcant effects on alcohol use and homelessness. Although smaller, the two coefŽcients roughly mirror those that are estimated under the sample selection strategy. But the coefŽcients that are attached to drug use are much smaller than those that are estimated under the sample selection strategy. The coefŽcients associated with the housing condition are also generally smaller under this strategy than under the sample selection strategy, which mayre•ect that the fact the predicted scores for the housing clients stem from a limited equation. Nevertheless, except for drug treatment outcomes, estimates are larger than those generated under the basic strategy.

The results in the next column of Table 7 stem from applying a supplemented strategy. This uses conventional control variable techniques to add to the basic equation all of the variables that predict acceptance in the sample selection model. As Table 7 notes, the estimated effects under this strategy generally are smaller than those calculated by sample selection models. Instead, the effects are similar to those that occur under the basic strategy, even if coefŽcients and levels of statistical signiŽcance are somewhat attenuated.

The results in the Žnal column of Table 7 stem from the propensity score matching strategy (Rosenbaum, 1986; Joffe & Rosenbaum, 1999). Under this strategy, experimental and ‘control’ cases are matched if they have similar scores on the traits that predict acceptance (that is, they are matched when the cases have similar propensities to accept). Estimates of the effects of treatment on outcomes are then conducted within the matched samples. Comparison cases that do not match up are dropped. Obviously, this strategy corrects accurately if acceptance only depends on the traits that are used to calculate the propensity scores; analyses use a sample that is matched on these traits. In other words, the strategy is based on a strong version of the assumption that the data on the outcomes of refusers are missing at random.

As is conventional, this paper uses probit equations to predict acceptance. It next matches acceptors to refusers on the basis of similar predicted scores from the ‘propensity’ equations (random sampling is used when there are duplicate matches).

86 Michael R. Sosin

Outcome equations then are estimated for the matched sample when otherwise replicating the basic strategy. Again, there is some awkwardness here. First, the propensity score, matching and outcome equations must be estimated separately by experimental condition. This re•ects the fact that the two conditions have different predictors of acceptance, so that they provide different ranges of propensity scores. Second, the clients in the housing condition are matched with clients in the control condition, while the clients in the case-management condition are matched with clients who reject that condition. This avoids using the same comparison cases in two sets of equations. The matching of acceptors and refusers is common under the propensity score strategy, in that it follows from the assumption that acceptors and refusers are equivalent when matched. Nevertheless, in the example (as is the case more generally), the samples of case-management acceptors and refusers do not always have identical propensity scores. ‘Near’ matches must be used, and this has an unknown impact on the results.15

As can be seen from Table 7, the estimates that are derived from the propensity score matching strategy are smaller than those calculated under the sample selection strategy. Instead, estimates are roughly comparable to the estimates that are derived from the supplemented (or basic) strategy, even if the coefŽcients are somewhat larger. The estimated effects of the intervention do not generallyremain statisticallysigniŽcant. This may re•ect the fact that the matching strategy reduces the sample size.

In sum, supplemented and propensity matching strategies, which share the assumption that the data for refusers are missing at random, supply relativelysimilar estimates. Both offer relatively mild degrees of bias correction compared to conventional (basic) analyses strategies. The instrumental variable and sample selection strategies, which do not rely on the assumption that data are missing at random, tend to provide larger estimates, particularly for the case-management condition. Perhaps the degree of bias correction is greater under the assumption that some of the reasons for refusal are unknown.

5. Conclusion

Despite these patterns, the most obvious conclusion is that varying correction strategies provide somewhat different estimates of the impact of the interventions. This means that the researcher cannot simply use the results from one correction strategy to represent all strategies. Rather, an argument must be made concerning which strategy to accept.

In the present case, the sample selection modelling strategyseems to be based on the relatively convincing and sensible assumption that acceptors and refusers differ in their reaction to the measured variables. Indeed, the comparisons of the baseline traits and outcomes of acceptors and refusers provide evidence for that assumption. Yet, somewhat different results occur under this and the instrumental variable strategy, even if both eschew the assumption that data are missing at random. This suggests that the other assumptions that are behind the strategies (such as that it is possible to control for selection) also matter. Unfortunately, these assumptions are difŽcult to assess and logically compare.

15A more advanced version calculates propensity scores, divides the entire sample into Ž ve groups based on these scores, and calculates outcomes within each such group. This technique seems to work better when the form of the equation predicting acceptance (the probit equation)is wrong, but it cannot be used here because the sample is too small (Rosenbaum, 1995; Joffe and Rosenbaum, 1999).

Some researchers also might argue that both strategies are more volatile and less trustworthy than strategies that make the conservative assumption that the data are missing at random, particularly given that both can considerably alter the size of coefŽcients. Unfortunately, it is rather difŽcult to decide this or to choose between strategies with the aid of data. For example, the assumptions behind any simulation can have a large impact on the comparative accuracy of correction strategies.

All this may be taken to suggest that researchers will be well served by comparing varying types of estimates, at least when the research stakes are high, or when analysts desire a little more information on the potential problems that might be caused by the pattern of refusal. If many such comparisons were available, researchers would at the very least have a better idea about whether alternate correction strategies frequently give rise to large differences in the estimates of experimental effects. More generally, researchers would be able to consider more fully the potential impacts of selective refusal, and whether these are as much a cause for statistical concern as appears to be the case for the example that is considered here.

Acknowledgements

This paper was prepared in part under grant number AA08773 from the National Institute of Alcohol Abuse and Alcoholism. The conclusions are those of the author. The author would like to acknowledge the useful comments provided by Susan Lambert, Melissa Roderick, and the anonymous reviewers.

References

Abel, M. H., & Cummings, P. (1993). A demonstration program for homeless male alcohol and other drug users. Journal of Mental Health Administration, 20, 90–99.

Baekeland, F., & Lundwall, L. (1975). Dropping out of treatment: Acritical review. Psychological Bulletin, 82, 738–783.

Bahr, H., & Caplow, T. (1974). Old men drunk and sober. New York: New York University Press. Barrow, S. M., Hellman, F., Lovell, A. M., Plapinger, J. D., Robinson, D. R., &Streuning, E. L. (1985).

Personal History Form. New York: New York State Psychiatric Institute, Community Support Systems Evaluation Program, Epidemiology of Mental Disorders Research Department.

Becker, M. H., & Maiman, L. A. (1975). Sociobehavioural determinants of compliance with health and medical care recommendations. Medical Care, 13, 10–24.

Berk, R. A. (1983). An introduction to sample selection bias in sociological data. American Sociological Review, 48, 386–398.

Blood, L., &Cornwall, A. (1994). Pretreatment variables that predict completion of an adolescent substance abuse treatment program. Journal of Nervous and Mental Disease, 182, 14–19.

Blumberg, L., Shipley, T. E., &Shandler, I. W. (1973). Skid Row and its alternatives. Philadelphia: Temple University Press.

Breen, R. (1996). Regression models: Censored, sample selected, or truncated data. Thousand Oaks, CA: Sage.

Burt, M. R. & Cohen, B. E. (1990). America’s homeless: Numbers, characteristics, and the programs that serve them. Washington, DC: Urban Institute Press.

Colson, P. (1990). Service use among the homeless. Unpublished doctoral thesis, School of Social Services Administration, University of Chicago.

Copeland, J., & Hall, W. (1992). A comparison of predictors of treatment drop-out of women seeking drug and alcohol treatment in a specialist women’s and two traditional mixed-sex treatment services. British Journal of Addiction, 87, 883–890.

88 Michael R. Sosin

Drake, R. E., McHugo, G. J., & Biesanz, J. (1995). The test–retest reliability of standardized instruments among homeless persons with substance use disorders. Journal of Studies of Alcohol, 56, 161–167.

Greene, W. (1991). LIMDEP Version 6.0: User’s manual and reference guide. Bellport, NY: Econometric Software, Inc.

Greene, W. H. (1997). Econometric analysis (3rd ed.). Upper Saddle River, NJ: Prentice Hall. Greene, W. H. (1992). Limdep Version 6.0. Bellport, NY: Econometric Software.

Grigsby, C., Baumann, D., Gregorich, S. E., & Roberts-Gray, C. (1990). DisafŽliation to entrenchment: Amodel for understanding homelessness. Journal of Social Issues, 46, 141–156.

Gueron, J. M., & Pauly, E. (1991). From welfare to work. New York: Russell Sage Foundation. Hartman, R. S. (1991). A Monte Carlo analysis of alternative estimators in models involving

selectivity. Journal of Business and Economic Statistics, 9, 41–49.

Hausman, J. A., &Wise, D. A. (1977). Social experimentation, truncated distributions, and efŽcient estimation. Econometrica, 54, 919–937.

Heckman, J. J. (1976). The common structure of statistical models of truncation, sample selection and limited dependent variables and a simple estimator for such models. Annals of Economic and Social Measurement, 5, 475–492.

Heckman, J. J. (1979). Sample selection bias as a speciŽcation error. Econometrica, 47, 153–161. Hickson, R., Mangione, T., Myers, A., & Scotch, N. (1982). Seeking help for drinking problems: A

study in the Boston metropolitan area. Journal of Studies on Alcohol, 43, 273–288. Hopper, K., & Baumohl, J. (1994). Held in abeyance: Rethinking homelessness and advocacy.

American Behavioral Scientists, 37, 522–552.

Jaccard, J., Turrisi, R., & Wan, C. K. (1990). Interaction effects in multiple regression. Newbury Park, CA: Sage.

Joffe, M. M., & Rosenbaum, P. R. (1999). Invited commentary: Propensity scores. American Journal of Epidemiology, 150, 327–333.

King, G. (1989). A seemingly unrelated Poisson regression model. Sociological Methods and Research, 17, 235–255.

Land, K. C., & McCall, P. L. (1993). Estimating the effect of nonignorable nonresponse in sample surveys: An application of Rubin’s Bayesian method for the estimation of community standards for obscenity. Sociological Methods and Research, 21, 291–316.

Little, R. J. A., & Rubin, D. B. (1987). Statistical analysis with missing data. New York: Wiley. Maddala, G. S. (1983). Limited-dependent and qualitative variables in econometrics. Cambridge:

Cambridge University Press.

Marlatt, G. A., & Gordon, J. R. (Eds.) (1985). Relapse prevention: Maintenance strategies in the treatment of addictive behaviours. New York: Guilford Press.

Manski, C. (1989). Anatomyof the selection problem. Journal of Human Resources, 29, 341–360. Manski, C. (1990). Nonparametric bounds on treatment effects. American Economic Review, 80,

319–323.

McLellan, A. T., Luborsky, L., Woody, G. E., & O’Brien, C. P. (1980). An improved diagnostic evaluation instrument for substance abuse patients: The Addiction Severity Index. Journal of Nervous and Mental Disease, 168, 26–30.

Moak, D. H., Anton, R. F., Malcolm, R., Randall, C. L., and Lantham, P. K. (1993). Alcoholic subjects with anxiety disorder: Characteristics of completers and noncompleters in a pharmacologic study. American Journal on Addictions, 2, 39–47.

Mohl, P. C., Martinex, D., Ticknor, C., & Appleby, J. (1989). Psychotherapy refusers. Comprehensive Psychiatry, 30, 245–250.

Moos, R. H. (1984). Context and coping: Toward a unifying conceptual framework. American Journal of Community Psychology, 12, 5–25.

Moos, R. H., & Finney, J. W. (1983). The expanding scope of alcoholism treatment evaluation.

American Psychologist, 38, 1035–1044.

Moos, R. H., King, M. J., & Patterson, M. A. (1996). Outcomes of residential treatment of substance abuse in hospital and community-based programs. Psychiatric Services, 47, 68–74.

Nelson, F. D. (1984). EfŽciency of the two-step estimator for models with endogenous sample selection. Journal of Econometrics, 24, 181–196.

Noel, N. E., McCrady, B. S., Stout, R. L., & Fisher-Nelson, H. (1987). Predictors of attrition from an outpatient alcoholism treatment program for couples. Journal of Studies on Alcohol, 48, 229–235.

Olkin, R., & Lemle, R. (1984). Increasing attendance in an outpatient alcoholism clinic: A comparison of two intake procedures. Journal of Studies on Alcohol, 45, 465–468.

Pekarik, G. (1983). Improvement in clients who have given different reasons for dropping out

Roffman, R. A., Klepsch, R., Wertz, J. S., Simpson, E. E., & Stephens, R. S. (1993). Predictors of attrition from an outpatient marijuana-dependence counseling program. Addictive Behaviors, 18, 553–566.

Rosenbaum, P. R. (1986). Dropping out of high school in the United States: An observational study.

Journal of Educational Statistics, 11, 207–224.

Rosenbaum, P. R. (1995). Quantiles in nonrandom samples and observational studies. Journal of the American Statistical Association, 90, 1424–1431.

Rosenstock, I. M. (1985). Understanding and enhancing patient compliance with diabetic regimens. Diabetes Care, 8, 610–616.

Rubin, D. B. (1977). Formalizing subjective notions about the effect of nonrespondents in sample surveys. Journal of the American Statistical Association, 72, 438–543.

Silberfeld, M., & Glaser, F. B. (1978). Use of the life table method in determining attrition from treatment. Journal of Studies on Alcohol, 39, 1582–1590.

Simpson, D. D. (1981). Treatment for drug abuse: Follow-up outcomes and length of time spent.

Archives of General Psychiatry, 38, 875–880.

Simpson, D. D., & Joe, G. W. (1993). Motivation as a predictor of early dropout from drug abuse treatment. Psychotherapy, 30, 357–368.

Sosin, M. R., Bruni, M., & Reidy, M. (1995). Paths and impacts in the progressive independence model: a homelessness and substance abuse intervention in Chicago. Journal of Addictive Diseases, 14, 1–20.

Sosin, M. R., Colson, P., &Grossman, S. (1988). Homelessness in Chicago: Social institutions and social change. Chicago: School of Social Service Administration, University of Chicago.

Sosin, M. R., Schwingen, J., & Yamaguchi, J. (1993). Case management and supported housing in Chicago: The interaction of program resources and client characteristics. Alcoholism Treatment Quarterly, 10, 35–50.

Sosin, M. R., Yamaguchi, J., Bruni, M., Grossman, S., Leonelli, B., Reidy, M., &Schwingen, J. (1994).

Treating homelessness and substance abuse in community context: A case management and supported housing demonstration. Chicago: School of Social Service Administration, University of Chicago.

Stahler, G. J., Shipley, T. E., Bartelt, D., Westcott, D., GrifŽths, E. E., &Shandler, I. (1993). Retention issues in treating homeless polydrug users: Philadelphia. Alcoholism Treatment Quarterly, 10, 201–216.

Stark, M. J. (1992). Dropping out of substance abuse treatment: A clinically oriented review.

Clinical Psychology Review, 12, 93–116.

Stolzenberg, R. M., & Relles, D. A. (1997). Tools for intuition about sample selection bias and its correction. American Sociological Review, 62, 494–507.

Stolzenberg, R. M., & Relles, D. A. (1990). Theory testing in a world of constrained research design: The signiŽcance of Heckman’s censored sampling bias correction for nonexperimental research. Sociological Methods and Research, 18, 395–415.

Tienda, M., Smith, S. A., & Ortiz, V. (1987). Industrial restructuring, gender segregation, and sex differences in earnings. American Sociological Review, 52, 195–210.

90 Michael R. Sosin

Tobin, J. (1958). Estimation of relationships for limited dependent variables. Econometrica, 26, 24–36.

Wainer, H. (Ed.) (2000). Drawing inferences from self-selected samples. Mahwah, NJ: Lawrence Erlbaum Associates.

Willis, R. J., &Rosen, S. (1979). Education and self-selection. Journal of Political Economy, 87(5), S7–S36.

Winship, C., & Mare, R. D. (1992). Models for sample selection bias. Annual Review of Sociology, 18, 327–350.

Received 28 April 2000; revised version received 16 January 2001

Appendix: Tobit selection term

The tobit selection term can be explained by Žrst noting that the tobit regressions are formulated in a similar way to sample selection corrections. Like the latter, they assume that, for y an underlying normally distributed outcome variable,

When scores are censored at 0, this equation is only directlyobservable for the manifest variable y, when y > 0:

Tobit models are unlike sample selection models because scores that are censored are predicted by the same independent variables that predict y. That is, these variables predict all scores for y, and thus predict the propensity to have an extreme score. Accordingly,

The last term is the inverse Mills ratio evaluated at b¢x i/j.

It follows that the sample selection equation for the tobit must consider the expected value for y under two conditions. Continuing with the symbol system

where d = ± 1/(1 ± r2)1/2 and F2 is the bivariate normal probability for (± h, ± k, r). This formula, while complex, is like that for the regression case in holding selection constant with a term that is a function of j, r, and the density and distribution functions of independent variables relevant to selection. It may be estimated using a maximum likelihood procedure.

It is possible to argue that the outcome equation should use a Poisson regression. This makes use of count data on the frequency of an event within a period, allowing the responses to increase nonlinearly across the distribution of the dependent variable (King, 1989). Atwo-stage sample selection correction has been developed by Terza (as cited in Greene, 1997). However, the current distribution is consistent with the use of tobits. Tobit models also allow for the application of maximum likelihood sample selection models.