Словари и журналы / Психологические журналы / p63British Journal of Mathematical and Statistical Psycholog

self-reported scores from the experiment’s second (last) follow-up interviews: the days clients report using alcohol in the 30-day period before the second follow-up interview; the days they report using drugs in the same period; and the days they report being homeless in 60 days. The interviews were conducted a year after programme entry. The outcome measures and the independent variables (which were also constructed from a baseline interview) generally have acceptable test–retest correlation coefŽcients.8

Independent variables

All outcome equations include as independent variables the two dummyor binary(zeroone) terms representing assignment into the intervention (treatment) conditions. This means that the coefŽcients that are attached to the two variables estimate the impact of what will be called the ‘interventions’. The coefŽcients implicitlycompare outcomes of individuals in each intervention condition to the outcomes of individuals in the control condition, who are provided a score of zero on both variables.

Control variables

All equations hold constant background variables, including the baseline level of the outcome variable, the interactions between the baseline score and each intervention, and various theoretically derived predictors (which differ by the dependent variable; Sosin et al., 1994). They also hold constant background variables that represent extraneous sources of change: the days between the baseline and follow-up interview; and the days of recent residence in situations ( jails, prisons and the like) in which substance abuse or homelessness is not possible. The baseline scores are centred to reduce the multicollinearity that occurs when using interaction terms ( Jaccard, Turrisi, & Wan, 1990).

Strategy and models

All outcome models use tobit regressions, which are appropriate because the outcome variables have large proportions of zero scores. In these tobit analyses, the impacts of the experiments are estimated under the assumption that clients who report outcome scores of zero may still differ in the propensity for substance abuse or homelessness.

The estimation strategies otherwise vary. The conventional basic strategy relies on the data on the acceptors. It predicts the outcomes by the experimental variables and the background variables. The intent-to-treat strategy adds data on the measured outcomes of refusers. It uses the same variables as does the basic strategy. The sample selection strategy relies on the same sample and variables as the basic strategy but adds the sample selection terms. Later sections of the paper discuss other correction strategies.

8Most of the items analysed here stem from the two instruments—the Addiction Severity Index (ASI; McLellan, Luborsky, Woody, & O’Brien, 1980) and the personal history form (PHF; Barrow, Hellman, Lovell, Plapinger, Robinson, & Streuning, 1985). The former is a 45-minute interview designed to determine substance impairment and related social and personal problems; the latter covers housing and demographic information. The reliability of the ASI and PHF has been established by test–retest analyses that utilized data from 20 clients enrolled in the current project, and about 20 from each of nine other homelessness and substance abuse projects funded by the National Institute for Alcohol Abuse and Alcoholism (n = 188 ; Drake, McHugo, & Biesanz, 1995). Intraclass correlation coefŽ cients, which compared baseline and retest responses (provided by the same clients about a week later), generally were above 0.70 on the ASI, and above 0.60 on the PHF. Chicago respondents tended to evince somewhat higher correlations.

74 Michael R. Sosin

4. Results of the application

4.1. Sample traits

Sample traits are reported in Table 1. Results near the top represent baseline traits; those at the bottom represent traits at the follow-up interview, which are the outcome scores.9 The baseline traits for the entire sample are reported at the top of the Žrst column. These roughly mirror the traits of a national sample of homeless adults (Burt & Cohen, 1990). The few differences include the greater proportion of current respondents who are black, and the slightly higher proportion who are women. Large city samples generally tend to have higher proportions of minority members and women, at least to some degree (Burt & Cohen, 1990; these results restrict responses to clients who are represented at both the baseline and second follow-up interview).10 Results reported at the bottom of the Žrst column further show that the outcomes generally improve over

time. This leads to the previously mentioned bunching of outcome scores at zero.

4.2. Group comparisons of traits

The data in the rest of Table 1 compare traits of subgroups of the sample, primarily to illustrate the possible existence of bias. Columns 2–4 thus report the selected traits of the acceptors of each condition; asterisks in the fourth column identify statistically signiŽcant differences between the traits of the three groups. Columns 5 and 6 report selected traits of the refusers of the two intervention conditions (no clients refused the control condition); asterisks in each of the two columns identity statistically signiŽcant differences between the traits of the acceptors and the refusers of a given intervention condition.

Results from the top part of columns 2–4, which mirror the searches for bias that are attempted by many researchers, do not indicate any obvious problem; there are no statistically signiŽcant differences in the three-way comparisons of the acceptors of the experimental conditions. However, the information on the refusers is of more concern. The asterisked quantities in column 5 suggest that housing refusers and acceptors differ on the reported days of homelessness at the time of the baseline interview, while those in column 6 suggest that case-management refusers and acceptors differ on the reported days of drug consumption at the time of the baseline interview. Despite the small sample of refusers, data in column 5 also suggests that housing refusers and acceptors differ to a nearly statistically signiŽcant degree on homeless history. As has been argued, differences between acceptors and refusers may cause bias in conventional calculations—even if the groups of acceptors demonstrate similar traits.

While not subjected to signiŽcance tests, results in the bottom portion of columns 5 and 6 suggest that there are differences in the outcomes between the control group and the refusers. This implies that the refusers are not a random sample of the original clients, in that their outcomes differ from those of untreated individuals who clearly comprise a random sample. Again, the implication is that biases may occur in analyses that exclude information on the refusers.

The results in Table 1 also verymodestly point to some of the difŽculties that emerge

9The statistical package used was LIMDEP (Econometric Software, Inc.).

10See Sosin, Colson, & Grossman (1988), Stahler et al. (1993) and Sosin et al. (1994). While this is not presented in the table, sample members also have the somewhat higher incomes that apparently typically characterize homeless clients found in programmes.

in the intent-to-treat analysis strategy—in particular, that this strategy is often based on the questionable assumption that little is changed by including the refusers as if they participated, since most have few symptoms and thus would not beneŽt from participation. Even if the results indicate the refuser’s baseline symptoms are often less severe than those of the acceptors, they also suggest that this is not always the case.

As might be expected, the outcome scores of the refusers are poorer than those of the acceptors, at least in Žve of the six comparisons. But while proponents of the intent-to-treat strategy normally argue that such outcomes are further evidence that refusers would not improve much if they participated (Simpson, 1981; Pekarik, 1983; Powell et al., 1987), a case can also be made that the failure to participate suppresses the experimental effects; refusers generally have less severe symptoms at the baseline interview but fare worse at the follow-up interview. In general, the data are open to debate concerning how the refusers would fare had theyparticipated. Further, while the data suggest that experimental clients fare better than control clients, it is not clear how refusal might affect these differences.

4.3. Outcome results: Three strategies

Basic strategy

Tables 2–4 compare results from the three strategies. In all of the represented equations, the coefŽcients that are attached to the experimental conditions show the calculated effects of the ‘interventions’ on the outcome variables. Technically speaking, the effect sizes are put in terms of ‘units’ because of the transformation of outcome variables in tobit regressions.

The Žrst column of each table reports results for the basic strategy. The results in Table 2 reveal that both the housing and case-management interventions reduce alcohol use. Those in Table 3 suggest that neither intervention has a statistically signiŽcant impact on drug use. Those in Table 4 show that the effect of the housing intervention on homelessness is large though not quite statistically signiŽcant, but that the effect of the case-management intervention on homelessness is statistically signiŽcant. These models suggest reasonable substantive results, in that there is other evidence that a case-management programme can outperform an inpatient programme (Moos, King, & Patterson, 1996).

In general, effect sizes are moderate. Depending on the outcome, they range between about 2 and 7 units for the housing intervention. CoefŽcients are between approximately 3 and 13 units for the case-management intervention.

Intent-to-treat strategy

The second columns of the tables reports analyses from the intent-to-treat strategy. In brief, the results continue to suggest the same statistically signiŽcant treatment effects—at least when using one-tailed tests of signiŽcance for these hypothesized relations. But the coefŽcients are attenuated. For example, Table 2 suggests that the estimate from the intent-to-treat analysis of the impact of the case management intervention on alcohol use is almost four units less than the estimate from the basic analysis.

As might be expected, estimates are suppressed because refusers generally do not fare as well as acceptors. This is illustrated in most of the Žndings from equations that predict the outcomes of refusers, alone, against those of the clients in the control

Note: *p < 0.05, **p < 0.01. For one-tailed test (intervention only), ² p < 0.05.

condition (when including the same control variables). These results suggest that, with respect to alcohol-use outcomes, housing refusers fare 2.7 units worse than control clients, and case-management clients fare only 4.3 units better; with respect to druguse outcomes, housing refusers fare 7.8 units better than control clients, but casemanagement refusers fare only 2.3 units better; with respect to homelessness outcomes, housing refusers fare 14.1 units worse, and case-management refusers fare only 6.6 units better. None of these Žndings is statistically signiŽcant at the 0.05 level.

Sample selection results: Outcome equations

The estimates that are calculated under the (maximum likelihood) sample selection analysis strategy are found in the last column of the three tables. For the sake of presentation, the sigmas (standard errors) for the sample selection models are reported separately from the sigmas for the other tobit analyses. We Žrst describe the results for the outcome equations and then discuss the selection equations.

The results for the outcome equations in part conŽrm the importance of the key statistically signiŽcant relations that were located by the basic strategy; the coefŽcients suggest that the housing and case-management interventions reduce alcohol use, and that the case-management intervention reduces homelessness. However, results reported in the last column of Table 3 also suggest that case-management intervention reduces drug use, while those in Table 4 show that housing intervention tends to reduce

Note: *p < 0.05, **p < 0.01. For one-tailed test (intervention only), ² p < 0.05.

intervention conditions and of two background traits (education and race). The other groups of variables stem from the previously mentioned theory of non-participation. Thus, the third group represents military experience and con•ict with others, measuring motivation in terms of traditional indicators of ‘disafŽliation’.

The fourth group of variables measures whether an individual was recruited after the recruitment worker began to fully detail the treatment component of the programme, after this worker began to stress the availabilityof recreational services, and during the high-energy Žrst 90 days of the programme. These variables represent the recruitment procedures that might affect the perceived costs and beneŽts of entering care. The Žfth group measures whether clients were recruited from a site that offered its own extended services, and whether clients previously received help in obtaining welfare beneŽts. These two variables represent access to alternative sources of help which, as mentioned earlier, might reduce the perceived beneŽts of the intervention. The Žnal group of variables measures professional referrals to programmes, and previous experience in drug and alcohol programmes. The two variables represent cues that may increase the perceived beneŽts of treatment. The selection equation excludes measures of baseline homelessness, substance abuse problems and mental health problems. These proved to be unpredictive once the other variables were entered.

As noted earlier, the included variables interact with the experimental conditions.

Note: *p < 0.05, **p < 0.01. For one-tailed test (intervention only), ² p < 0.05.

Few of these interactions involve the housing condition, which has very little refusal to explain.

Selection model: Results

The remaining columns of Table 5 report the results of the selection models. There are three such models since each of the three outcome equations must be estimated simultaneouslyand separatelywith the selection equation (Breen, 1996). From the table it can be seen that each of the three models locates at least one signiŽcant predictor of acceptance. Each also includes other large coefŽcients. All this conŽrms that acceptance is not fully random.

As might be expected, the coefŽcients are similar in size across the three selection models. Nevertheless, there are some differences in statistical signiŽcance, particularly in the interaction terms involving the case-management intervention. For example, in the selection model that is estimated with the outcome equation for alcohol use, there