becomes: Y = b0 + b1Age + b2D1 +b3D2. For example, if the subject is a current smoker, D1 = 1 and D2 = 0; if an ex-smoker, D1 = 0 and D2 = 1; if a non-smoker, D1 = 0 and D2 = 0. Notice in the last situation that the smoking status variable effectively disappears from the model.

This coding scheme can be extended to deal with nominal variables with any reasonable number of categories, depending on the sample size.5 The simplest situation is a nominal variable with only two categories, such as sex, which can be represented by one design variable with values 0 (if male) or 1 (if female).

Exercise 17.5 The first three subjects in the study of systolic blood pressure and its relationship with age and smoking status are, a 50-year-old smoker, a 55-year-old nonsmoker and a 35-year-old ex-smoker, respectively. Fill in the first three rows of the data sheet shown in Table 17.3, as appropriate.

Table 17.3 Data sheet for systolic blood pressure relationship

1

2

3

Model building and variable selection

At the beginning of this chapter we chose body mass index as the variable to explain or model systolic blood pressure. In practice, researchers may or may not have an idea about which variables they think are relevant in explaining the variation in their dependent variable. Whether they do or they don’t will influence their decision as to which variables to include in their model, i.e. their variable selection procedure.

There are two main approaches to the model-building process:

First, automated variable selection – the computer does it for you. This approach is perhaps more appropriate if you have little idea about which variables are likely to be relevant in the relationship.

Second, manual selection – you do it! This approach is more appropriate if you have a particular hypothesis to test, in which case you will have a pretty good idea which independent

5As a rule of thumb, you need at the very least 15 subjects for each independent variable in your model. If you’ve got, say, five ordinal and/or metric independent variables in your model, you would need a minimum of 75 subjects. If you want also to include a single nominal variable with five categories (i.e. four design variables), you would need another 60 subjects. In these circumstances, it might help to amalgamate some categories.

variable is likely to be the most relevant in explaining your dependent variable. However, you will almost certainly want to include other variables to control for confounding (see p. 81 for an account of confounding).

Both of these methods have a common starting procedure, as follows:6

Identify a list of independent variables that you think might possibly have some role in explaining the variation in your dependent variable. Be as broad-minded as possible here.

Draw a scatterplot of each of these candidate variables (if it is not a nominal variable), against the dependent variable. Examine for linearity. If any of the scatterplots show a strong, but not a linear relationship with the dependent variable, you will need to code them first before entering them into the computer data sheet. For example, you might find that the relationship between the dependent variable and ‘age’ is strong but not linear. One approach is to group the age values into four groups, using its three quartile values to define the group boundaries, and then code the groups with three design variables.

Perform a series of univariate regressions, i.e. regress each candidate independent variable in turn against the dependent variable. Note the p-value in each case.

At this stage, all variables that have a p-value of at least 0.2 should be considered for inclusion in the model. Using a p-value less than this may fail to identify variables that could subsequently turn out to be important in the final model.

With this common starting procedure out of the way, we can briefly describe the two variable selection approaches, starting with automated methods.

Automated variable selection methods

Forwards selection: The program starts with the variable that has the lowest p-value from the univariate regressions. It then adds the other variables one at a time, in lowest p-value order, regressing each time, retaining all variables with p-values < 0.05 in the model.

Backwards selection: The reverse of forwards selection. The program starts with all of the candidate variables in the model, then the variable that has highest p-value > 0.05, is removed. Then the next highest p-value variable, and so on, until only those variables with a p-value < 0.05 are left in the model, and all other variables have been discarded.

Forwards or backwards stepwise selection: After each variable is added (or removed), the variables which were already (or are left) in the model are re-checked for statistical significance; if no longer significant they are removed. The end result is a model where all variables have a p-value < 0.05.

6Note that the criteria used by the different computer regression programs to select and de-select variables differ.

These automated procedures have a number of disadvantages, although they may be useful when researchers have little idea about which variables are likely to be relevant. As an example of the automated approach, the authors of a study into the role of arginase in sickle cell disease, in which the outcome variable was log10 arginase activity (Morris et al. 2005), comment:

This modelling used a stepwise procedure to add independent variables, beginning with the variables most strongly associated with log10arginase with P ≤ 0.15. Deletion of variables after initial inclusion in the model was allowed. The procedure continued until all independent variables in the final model had P ≤ 0.05, adjusted for other independent variables, and no additional variables had P ≤ 0.05.

Manual variable selection methods

Manual, DIY methods, are often more appropriate if the investigators know in advance which is likely to be their principal independent variable. They will include this variable in the model, together with any other variables that they think may be potential confounders. The identity of potential confounders will have been established by experience, a literature search, discussions with colleagues and patients and so on. There are two alternative manual selection procedures:

Backward elimination: The main variable plus all of the potentially confounding variables are entered into the model at the start. The results will then reveal which variables are statistically significant (p-value < 0.05). Non-significant variables can then be dropped, one at a time in decreasing p-value order, from the model, regressing each time. However, if the coefficient of any of the remaining variables changes markedly7 when a variable is dropped, the variable should be retained, since this may indicate that it is a confounder.

Forward elimination: The main explanatory variable of interest is put in the model, and the other (confounding) variables are added one at a time in order of (lowest) p-value (from the univariate regressions). The regression repeated each time a variable is added. If the added variable is statistically significant it is retained, if not it is dropped, unless any of the coefficients of the existing variables change noticeably, suggesting that the new variable may be a confounder.

The end result of either of these manual approaches should be a model containing the same variables (although this model may differ from a model derived using one of the automated procedures). In any case, the overall objective is parsimony, i.e. having as few explanatory variables in the model as possible, while at the same time explaining the maximum amount of variation in the dependent variable. Parsimony is particularly important when sample size is on the small side. As a rule of thumb, researchers will need at least 15 observations for each independent variable to ensure mathematical stability, and at least 20 observations to obtain reasonable statistical reliability (e.g. narrow-ish confidence intervals).

As an example of the manual backwards selection approach, the authors of a study of birthweight and cord serum EPA concentration (Grandjean et al. 2000), knew that cord serum

7There is no rule about how big a change in a coefficient should be considered noteworthy. A value of 10 per cent has been suggested, but this seems on the small side.

EPA was their principal independent variable, and but wanted to include possible confounders in their model. They commented:

Multiple regression analysis was used to determine the relevant importance of predictors of the outcome (variable). Potential confounders were identified on the basis of previous studies, and included maternal height and weight, smoking during pregnancy, diabetes, parity, gestational length, and sex of the child. Covariates8 were kept in the final regression equation if statistically significant (p < 0.01) after backwards elimination.

Incidentally, the main independent variable, cord serum concentration, was found to be statistically significant (p-value = 0.037), as were all of the confounding variables.

Goodness-of-fit again: R¯2

When you add an extra variable to an existing model, and want to compare goodness-of-fit

don’t need to concern us here, but R2 will increase when an extra independent variable is added

¯2 to the model, without there necessarily being any increase in explanatory power. However, if R

increases, then you know that the explanatory power (its ability to explain more of the variation

¯2 = in the dependent variable) of the model has increased. From Figure 17.3 or Figure 17.4, R

0.613 in the simple regression model with only hip circumference as an independent variable.

¯2

From Figure 17.5, with both hip and waist circumferences included, R increases to 0.635, so this multiple regression model does show a small but real improvement in goodness-of-fit, and would be preferred to either of the simple regression models. Of course, you might decide to explore the possibility that other independent variables might also have a significant role to play in explaining variation in body mass index; age is one obvious contender, as is sex, and should be included in the model.

Exercise 17.6 Table 17.4 contains the results of a multiple linear regression model from a cross-section study of disability, among 1971 adults aged 65 and over in 1986 (Kavanagh and Knapp 1998). The objective of the study was to examine the utilisation rates of general practitioners’ time by elderly people resident in communal establishments. The dependent variable was the natural log of weekly utilisation (minutes) per resident.9 There were 10 independent variables, as shown in the figure.

(a) Identify those independent variables whose relationship with the dependent variable is statistically significant. (b) What is the effect on the natural log of utilisation time, and what is this in actual minutes, if there is an increase of: (i) one person in the number in a

8i.e. independent variables.

9Probably because the researchers believed the utilisation rate to be skewed. See Figure 5.6 for an example of transformed data.

private residential home; (ii) one unit in the severity of disability score? (c) How much of the variation in general practitioners’ utilisation time is explained by the variation in the independent variables?

Table 17.4 Sample regression coefficients from a linear regression model, where the dependent variable is the natural log of the utilisation time (minutes) of GPs, by elderly patients in residential care. The independent variables are as shown. Reproduced from BMJ, 317, 322–7, courtesy of BMJ Publishing Group

R2 = 0.1098, F(17,415) = 9.71, P = <0.0001. Sample size = 1971 in 433 sampling units.

Adjustment and confounding

One of the most attractive features of the multiple regression model is its ability to adjust for the effects of possible association between the independent variables. It’s quite possible that two or more of the independent variables will be associated. For example, hip (HIP) and waist (WST) circumference are significantly positively associated with r = +0.783 and p-value = 0.000. The consequence of such association is that increases in HIP are likely to be accompanied by increases in WST. The increase in HIP will cause bmi to increase both directly, but also indirectly via WST. In these circumstances it’s difficult to tell how much of the increase in bmi is due directly to an increase in HIP, and how much to the indirect effect of an associated increase in WST.

The beauty of the multiple regression model is that each regression coefficient measures only the direct effect of its independent variable on the dependent variable, and controls or adjusts for any possible interaction from any of the other variables in the model. In terms of the results in Figure 17.5, an increase in HIP of 1 cm will cause mean bmi to increase by 0.261 kg/m2 (the value

Table 17.5 Multiple regression results from a cross-section study into the relationship between bone lead and blood lead levels and the development of hypertension in 512 individuals selected from a cohort study. The figure show the outcome from three multiple linear regression models, with systolic blood pressure as the dependent variable. Reproduced from Amer. J. Epid., 153, 164–71, courtesy of Oxford University Press

† Parameter estimates are based on 1 standard deviation (SD) in blood lead level (4.03 μg/dl), tibia lead level (13.65 μg/g), and patella lead level (19.55 μg/g).

dependent variable; a linear relationship between the dependent and each independent variable; error terms with constant spread and Normally distributed; and the independent variables not perfectly correlated with each other. How can we check that these assumptions are satisfied?

A metric continuous dependent variable. Refer to Chapter 1 if you are unsure how to identify a metric continuous variable.

A linear relationship between the dependent variable and each independent variable. Easiest to investigate by plotting the dependent variable against each of the independent variables; the scatter should lie approximately around a straight line.11 The other possibility is to plot the residual values against the fitted values of the independent variable (bmi in our example). These are the values the estimated regression equation would give for mean bmi, for every combination of values of HIP and WST. The scatter should be evenly spread around zero, with no discernible pattern, such as in Figure 17.7(a).

The residuals have constant spread across the range of values of the independent variable. Check with a plot of the residual values against the fitted values of bmi. The spread of the residuals should be fairly constant around the zero value, across the range of fitted values of the independent variable. Figure 17.7(b) is an example of non-constant variance. The spread appears to increase as the value of the independent variable increases. Figure 17.7(c) is an example of both non-linearity and non-constant variance.

The residuals are Normally distributed for each fitted value of the independent variable. This assumption can be checked with a histogram of the residuals. For our bmi example, the histogram in Figure 17.10 indicates that, apart from a rather worrying outlier, the distribution is Normal. You might want to identify which woman this outlier represents and check her data for anomalies.

The independent variables are not perfectly correlated with each other. Unfortunately, this is not an easy assumption to check. Some degree of correlation is almost certain to exist among some of the independent variables.

Exercise 17.8 (a) Explain briefly each of the basic assumptions of the multiple linear regression model. (b) With the aid of sketches where appropriate, explain how we can test that these assumptions are satisfied.

11Notice that we only have to establish this property of linearity for the metric independent variables in the model. Any binary variables are linear by default – they only have two points, which can be joined with a straight line. Any ordinal independent variables will have to be expressed as binary dummies – again linear by default for the same reason.