12

Testing hypotheses about the difference between two population parameters

Learning objectives

When you have finished this chapter you should be able to:

Explain how a research question can be expressed in the form of a testable hypothesis. Explain what a null hypothesis is.

Summarise the hypothesis test procedure.

Explain what a p-value is.

Use the p-value to appropriately reject or not reject a null hypothesis.

Summarise the principal tests described in this chapter, along with their most appropriate application, and any distributional and other requirements.

Interpret SPSS and Minitab results from a hypothesis test.

Interpret published results of hypothesis tests.

Point out the advantages of confidence intervals over hypothesis tests. Describe type I and type II errors, and their probabilities.

Medical Statistics from Scratch, Second Edition David Bowers

C 2008 John Wiley & Sons, Ltd

142 CH 12 TESTING HYPOTHESES ABOUT THE DIFFERENCE BETWEEN TWO POPULATION PARAMETERS

Explain the power of a test and how it is calculated.

Explain the connection between power and sample size.

Calculate sample size required in some common situations.

The research question and the hypothesis test

The procedures discussed in the preceding three chapters have one primary aim: to use confidence intervals to estimate population parameter values, and their differences and ratios. We were able to make statements like, ‘We are 95 per cent confident that the range of values defined by the confidence interval will include the value of the population parameter,’ or, ‘The confidence interval represents a plausible range of values for the population parameter.’

There is, however, an alternative approach called hypothesis testing, which uses exactly the same sample data as the confidence interval approach, but focuses not on estimating a parameter value, but on testing whether its value is the same as a previously specified or hypothesised value. In recent years, the estimation approach has become more generally favoured, primarily because the results from a confidence interval provides more information than the results of a hypothesis test (as you will see a bit later). However, hypothesis testing is still very common in research publications, and so I will describe a few of the more common tests.1 Let’s first establish some basic concepts.

The null hypothesis

As we have seen, almost all clinical research begins with a question. For example, is Malathion a more effective drug for treating head lice than d-phenothrin? Is stress a risk factor for breast cancer? To answer questions like this you have to transform the research question into a testable hypothesis called the null hypothesis, conventionally labelled H0. This usually takes the following form:

H0: Malathion is NOT a more effective drug for treating head lice than d-phenothrin.

H0: Stress is NOT a risk factor for breast cancer.

Notice that both of these null hypotheses reflect the conservative position of no difference, no risk, no effect, etc., hence the name, ‘null’ hypothesis. To test this null hypothesis, researchers will take samples and measure outcomes, and decide whether the data from the sample provides strong enough evidence to be able to refute or reject the null hypothesis or not. If evidence against the null hypothesis is strong enough for us to be able to reject it, then we are implicitly accepting that some specified alternative hypothesis, usually labelled H1, is probably true.

1 And there are some situations where there is no reasonable alternative to a hypothesis test.

The hypothesis testing process

The hypothesis testing process can be summarised thus:

Select a suitable outcome variable.

Use your research question to define an appropriate and testable null hypothesis involving this outcome variable.

Collect the appropriate sample data and determine the relevant sample statistic, e.g. sample mean, sample proportion, sample median, (or their difference or ratio), etc.

Use a decision rule that will enable you to judge whether the sample evidence supports or does not support your null hypothesis.

Thus, on the strength of this evidence, either reject or do not reject your null hypothesis.

Let’s take a simple example. Suppose you want to test whether a coin is fair, i.e. not weighted to produce more heads or more tails than it should. Your null hypothesis is that the coin is fair, i.e. will produce as many heads as tails, so that the population proportion π , equals 0.5. Your outcome variable is the sample proportion of heads, p. You toss the coin 100 times, and get 42 heads, so p = 0.42. Is this outcome compatible with your hypothesised value of 0.5? Is the difference between 0.5 and 0.42 statistically significant or could it be due to chance?

You can probably see the problem. How do we decide what proportion of heads we might expect to get if the coin is fair? As it happens, there is a generally accepted rule, which involves something known as the p-value.

The p-value and the decision rule

The hypothesis test decision rule is: If the probability of getting the number of heads you get (or even fewer) is less than 0.05,2 when the null hypothesis is true, then this is strong enough evidence against the null hypothesis and it can be rejected. The beauty of this rule is that you can apply it to any situation where the probability of an outcome can be calculated, not just to coin tossing.

As a matter of interest, the probability of getting say 42 or fewer heads if the coin is fair is 0.0666, which is not less than 0.05. This is not strong enough evidence against the null hypothesis. However, if you had got 41 heads or fewer, the probability of which is 0.0443, this is less than 0.05, now the evidence against H0 is strong enough and it can be rejected. The coin is not fair. This crucial threshold outcome probability (0.0443 in this example), is called the p-value, and defined thus:

A p-value is the probability of getting the outcome observed (or one more extreme), assuming the null hypothesis to be true.

2 Or 0.01. There is nothing magical about these values, they are quite arbitrary.

144 CH 12 TESTING HYPOTHESES ABOUT THE DIFFERENCE BETWEEN TWO POPULATION PARAMETERS

So, in the end, the decision rule is simple:

Determine the p-value for the output you have obtained (using a computer).

Compare it with the critical value, usually 0.05.

If the p-value is less than the critical value, reject the null hypothesis; otherwise do not reject it.

When you reject a null hypothesis, it’s worth remembering that although there is a probability of 0.95 that you are making the correct decision, there is a corresponding probability of 0.05 that your decision is incorrect. In fact, you never know whether your decision is correct or not,3 but there are 95 chances in 100 that it is. Compare this with the conclusion from a confidence interval where you can be 95 per cent confident that a confidence interval will include the population parameter, but there’s still a 5 per cent chance that it will not.

It’s important to stress that the p-value is not the probability that the null hypothesis is true (or not true). It’s a measure of the strength of the evidence against the null hypothesis. The smaller the p-value, the stronger the evidence (the less likely it is that the outcome you got occurred by chance). Note that the critical value, usually 0.05 or 0.01, is called the significance level of the hypothesis test and denoted α (alpha). We’ll return to alpha again shortly.

Exercise 12.1 Suppose you want to check your belief that as many males as females use your genito-urinary clinic. (a) Frame your belief as a research question. (b) Write down an appropriate null hypothesis. (c) You take a sample of 100 patients on Monday and find that 40 are male. The p-value for 40 or fewer males from a sample of 100 individuals is 0.028. Do you reject the null hypothesis? (d) Your colleague takes a sample of 100 patients on the following Friday and gets 43 males, the p-value for which is 0.097. Does your colleague come to the same decision as you did? Explain your answer.

A brief summary of a few of the commonest tests

Some hypothesis tests are suitable only for metric data, some for metric and ordinal data, and some for ordinal and nominal data. Some require data to have a particular distribution (often Normal); these are parametric tests. Some have no or less strict distributional requirements; the non-parametric tests. Before I discuss a few tests in any detail, I have listed in Table 12.1 a brief summary of the more commonly used tests, along with their data and distributional requirements, if any. I am ignoring tests of single population parameters since these are not required often enough to justify any discussion.

3 Because you’ll never know what the value of any population parameter is.