Statistical Inference  73

Note that the estimated F statistic for rehabilitative therapy, F = 11.44, does exceed the critical value for α < 0.05 (actually, α is <0.0001); thus, we conclude that the samples for the three different rehabilitative therapies represent populations with different means. In other words, the different rehabilitative therapies produce different days of recovery. However, the estimated F statistic for the drug therapy does not exceed the critical table F value for α = 0.05; thus, we accept the null hypothesis that drug therapy does not significantly affect the days of recovery and that the samples are not drawn from populations with different means. Moreover, the third F statistic, F = 2.84, is greater than the table F value for α < 0.05, which suggests that there may be a significant difference in days of recovery due to an interaction between rehabilitative therapy and drug therapy.

5.3.3 Tukey’s Multiple Comparison Procedure

Once we have established that there is a significant difference in means across treatments within a factor, we may use post hoc tests, such as the Tukey’s HSD multicomparison pairwise test [3, 9]. ANOVA simply shows that there is at least one treatment mean that differs from the others. However, ANOVA does not provide information on specifically which treatment mean(s) differs from which treatment mean(s). The Tukey’s HSD test allows us to compare the statistical differences in means between all pairs of treatments. For a one-factor experiment with k treatments, there are k(k − 1) /2 pairwise comparisons to test. The important point to note is that if α is the probability of a type I error for one comparison, the probability of making at least a type I error for multiple comparisons is much greater. So, if we want (1 − α) ×100% confidence for all possible pairwise comparisons, we must start with a much smaller α. Tukey’s multiple comparison procedure allows for such an adjustment in significance when performing pairwise comparisons.

• • • •

75

c h a p t e r 6

Linear Regression and

Correlation Analysis

Oftentimes, in biomedical engineering research or design, we are interested in whether there is a correlation between two variables, populations, or processes. These correlations may give us information about the underlying biological processes in normal and pathologic states and ultimately help us to model the processes, allowing us to predict the behavior of one process given the state of another correlated process.

Given two sets of samples, X and Y, we ask the question, “Are two variables or random processes, X and Y, correlated?” In other words, can y be modeled as a linear function of x such that

yˆ = mx + b.

Look at the following graph of experimental data (Figure 6.1), where we have plotted the data set, yi, against the data set, xi:

We note that the data tend to fall on a straight line. There tends to be a trend such that y increases in proportion to increases in x. Our goal is to determine the line (the linear model) that best fits these data and how close the measured data points lie with respect to the fitted line (generated by the model). In other words, if the modeled line is a good fit to the data, we are demonstrating that y may be accurately modeled as a linear function of x, and thus, we may predict y given x using the linear model.

The key to fitting a line that best predicts process y from process x, is to find the parameters m and b, which minimize the error between model and actual data in a least-squares sense:

min [(y − yˆ)2].

In other words, as illustrated in Figure 6.2, for each measured value of the independent variable, x, there will be the measured value of the dependent variable, y, as well as the predicted or

Linear Regression and Correlation Analysis  77

modeled value of y, denoted as ŷ, that one would obtain if the equation yˆ = mx + b is used to predict y. The equation ei = yi − yˆi denotes the errors that occur at each pair of (xi,yi) when the modeled value does not exactly align with the predicted value because of factors not accounted for in the model (noise, random effects, and nonlinearities).

In trying to fit a line to the experimental data, our goal is to minimize these ei between the measured and predicted values of the dependent variable, y. The method used in linear regression and many other biomedical modeling techniques is to find model parameters, such as m and b, that minimize the sum of squared errors, ei2.

For linear regression, we seek a least-squares estimate of m and using the following approach:

Suppose we have N samples each of processes x and y. We try to predict y from measured x using the following model:

yˆ = mx + b.

The error in prediction at each data point, xi, is

errori = yi − yˆi.

In the least-squares method, we choose m and b to minimize the sum of squared errors:

To find a closed form solution for m and b, we can write an expression for the sum of squared

errors:

We then replace yˆi with (mxi + b), our model, and carry out the squaring operations [3, 5]. We can then take derivatives of the above expression with respect to m and then again with re-

spect to b. If we set the derivative expressions to zero to find the minimums, we will have two equations in two unknowns, m and b, and we can simply use algebra to solve for the unknown parameters, m and b. We will get the following expressions for m and b in terms of the measured xi and yi: