Ординатура / Офтальмология / Английские материалы / Using and Understanding Medical Statistics_Matthews, Farewell_2007

Table 2.1. A 2 ! 2 contingency table summarizing the data described in Thomas et al. [1]

the significance level (‘p-value’) of the data with respect to the hypothesis. Of course, there are assumptions and calculations which are an implicit part of the evaluation process. In fact, there are six features which are common to all significance tests, and §2.3 is chiefly a discussion of these features. But first, we propose to consider a specific example, and to explain the assumptions and arguments which give rise to the significance level of a test and the resulting conclusion.

2.2. An Example

Thomas et al. [1] discuss relapse patterns in irradiated Wilms’ tumor patients. In particular, they describe a review of radiotherapy records which indicated that 23 out of 259 patients were judged to have been treated with a radiation field of inadequate size. An obvious hypothesis of interest is the suggestion that inadequate radiotherapy is associated with a higher risk of relapse in the operative site. Subsequent investigation of these 259 patients revealed that there were four relapses in the operative site, two of which occurred in the 23 patients who received inadequate radiotherapy. The data may be concisely presented in a summary table containing two rows and two columns (see table 2.1); tables of this form are usually called 2 ! 2 contingency tables.

A major purpose of this study of 259 patients was to determine whether inadequate radiotherapy is associated with a higher risk of relapse in the operative site. If we examine the data, we see that the relapse rate among patients whose field size was adequate was 2/236 (0.9%), whereas the corresponding figure for patients who received inadequate radiotherapy was 2/23 (8.7%). If Thomas et al. were only interested in the hypothesis concerning relapse in the operative site for the 259 patients studied, then the data indicate that inadequate field size is associated with an elevated relapse rate in those 23 patients. However, this is not the question which the researchers want to answer. They are interested in the population of patients who are treated for Wilms’ tumor.

Table 0

Fig. 2.1. The five possible 2 ! 2 tables with fixed marginal totals 4, 255, 23 and 236.

The 259 patients in the study constitute a sample from that population. On the basis of the data collected from the sample, Thomas et al. wish to infer whether the relapse rate in the operative site is higher for all patients in the population who receive inadequate radiotherapy.

In order to arrive at any conclusions regarding the hypothesis of an elevated risk of relapse based on the sample data, we must assume that the sample is truly representative of the population in all pertinent respects. However, even if we assume that this is true, how should the results of the study be interpreted? Should we conclude that the sample relapse rates (0.009 and 0.087) are the same as the population relapse rates, and therefore inadequate field size is associated with a ten-fold increase in the risk of relapse? Surely not! After all, the sample size is fixed at 259 patients, 23 of whom received inadequate radiotherapy, and only four relapses in the operative site were observed. If we adjust the numbers in the four cells of the 2 ! 2 table, always taking care to maintain the same row and column totals, then we see that only five different 2 ! 2 tables

Table 2.2. Sample relapse rates in tables 0 through 4 of figure 2.1

could possibly be observed in a study of 259 patients with these relapse and treatment totals. These five tables (labelled 0 through 4 for convenience) are displayed in figure 2.1; notice that the label for each 2 ! 2 table corresponds to the number in the upper left-hand corner. The sample relapse rates for each of these tables are indicated in table 2.2.

Table 2.2 illustrates the folly of assuming that sample relapse rates are the same as population relapse rates. Quite obviously, the fixed study totals (259 patients, 4 relapses and 23 patients receiving inadequate radiotherapy) severely restrict the set of possible sample relapse rates that may be observed to exactly five pairs of values.

This problem with sample relapse rates suggests that perhaps we should redirect our attention from the sample to the population. Clearly, there is a population relapse rate, r1 say, for patients who received adequate radiotherapy, and there is a second relapse rate, r2 say, for patients whose radiotherapy was inadequate. The hypothesis which Thomas et al. wish to investigate is whether or not these relapse rates are the same, i.e., whether r1 = r2 or r1 0 r2. If r1 and r2 are identical, or very similar, then we might expect this similarity to be reflected in the observed data, to the extent that the sample size, number of relapses and treatment totals permit. On the other hand, if r1 and r2 are quite different, then we would expect the data to reflect this difference, again, as the numbers permit.

If we suppose that r1 and r2 are the same, then a natural estimate of the common relapse rate r = r1 = r2, say, is 4/259 = 0.015, the overall relapse rate in the sample. Of course, r is unlikely to be exactly 0.015; nevertheless, if our sample is representative of the population, r should be fairly close to the observed relapse rate. And if 0.015 is close to r, the relapse rate in the population of Wilms’ tumor patients, then among the 23 who received inadequate radiotherapy we would expect to observe approximately 23 ! 0.015 = 0.355 relapses in the operative site. From this perspective we realize that there is an obvious ranking for tables 0 through 4, based on the observed number of relapses among the 23 patients who received inadequate radiotherapy. If the population relapse rates are the same, i.e., r1 = r2, then table 0 is most consistent with that

Table 2.3. The probability of observing each of tables 0 through 4 if r1 = r2

situation, table 1 is less consistent than table 0 and so on; table 4 is clearly the least consistent of all five possible sample outcomes. To express this idea in a somewhat different way, we might say that, if r1 = r2, then we would be most inclined to expect table 0 to result from the review of patient records, and least inclined to expect table 4.

Once we have realized this fact, we have begun to develop an objective method of evaluating the degree of consistency between the data and the hypothesis of interest. Nevertheless, we still have a considerable distance to cover. For example, it should be clear that because the study patients are a sample from the population, and because sampling necessarily involves uncertainty, we cannot state conclusively that, if r1 = r2, the study records would not yield table 4. Quite obviously, what we need now is an objective measure of how likely, or unlikely, each of tables 0 through 4 would be if the relapse rates r1 and r2 are the same. And this is precisely what probability calculations can provide. If we assume, a priori, that r1 = r2, then it is fairly simple for a statistician to calculate the probability that in a sample of 259 patients, 23 of whom received inadequate radiotherapy and 4 of whom relapsed at the operative site, exactly 0 (or 1, 2, 3 or 4) patients who relapsed also received inadequate radiotherapy. Rather than labor over the details of these calculations, we have listed the probabilities in table 2.3 and will defer the details to chapter 3.

The entries in table 2.3 indicate that, if the population relapse rates for both groups of Wilms’ tumor patients are the same, then samples of size 259 having the same relapse and treatment totals (4 and 23, respectively) are most likely to yield table 0 and least likely to result in table 4. However, each of the five tables might possibly occur in a particular sampling situation. Notice, also, that table 2 (which was reported by Thomas et al.) is a relatively unlikely outcome if the equal relapse rates assumption is correct. If we consider all tables which are no more consistent with the hypothesis r1 = r2 than table 2, i.e., tables 2, 3 and 4, then by adding the corresponding probabilities we determine that data which are no more consistent with the hypothesis r1 = r2 than those which were observed (i.e., tables 2, 3 and 4) could be expected to occur in approximately four studies out of every hundred having the same sample size (259),

Table 2.4. The two possible explanations for the observed result in the study of Thomas et al. [1]

relapse and treatment totals (4, 23, respectively). It is this value, 0.041, the sum of the probabilities for tables 2, 3 and 4, that statisticians call the significance level (p-value) of a test of the hypothesis r1 = r2.

At this point in the argument we pause to consider possible explanations for the observed result. Each explanation is based on a specific hypothesis about the population relapse rates. Clearly, to be satisfactory, an explanation should account for the significance level of the data. As table 2.4 indicates, there are basically two explanations which account for the result observed in the sample of Thomas et al.

Clearly, we cannot determine which explanation, if either, is correct; this is beyond the realm of statistics. We cannot determine directly whether Explanation 1 is more likely since, if r1 0 r2, we do not know how great is the actual difference in relapse rates. However, we do know that if Explanation 1 is correct, the significance level (p = 0.04) specifies how likely we would be to obtain a study sample which is no more consistent with the hypothesis of equal relapse rates than table 2.

Which explanation is preferable, the first or the second? As a general rule in scientific research, the simplest description or model of a situation which is, at the same time, consistent with the available evidence is preferred. On this basis, Explanation 1 usually would be selected, a priori, because it does not involve a relapse rate which depends on the radiotherapy received. However, if we adopt Explanation 1, then we are also choosing to account for the observed difference in sample relapse rates on the basis of sampling uncertainty. We have calculated that such discrepancies would occur in at most four studies out of every hundred of a similar size. Statisticians normally suggest that when the significance level associated with the simpler explanation is very small, it is justifiable to conclude that the simpler explanation is not consistent with the evidence at hand; instead, the more complicated explanation should be adopt-

ed. Therefore, since the significance level of the data of Thomas et al. with respect to the hypothesis of equal relapse rates is relatively small (p = 0.04), we conclude from an analysis of the study records that the weight of evidence contradicts the hypothesis that r1 = r2, i.e., Explanation 2 is preferred.

2.3. Common Features of Significance Tests

The preceding example provides a valuable introduction to the general topic of significance tests. The purpose of a significance test is to determine the degree of consistency between a specific hypothesis, represented by H say, and a set of data. In most cases, H is a simple description of some aspect of a particular population and the collected data are obtained from a sample drawn from the population. While the sample is thought to be representative of the population, it is clear that a different sample would give rise to a different set of data. There are always two fairly obvious explanations for the observed results:

IThe hypothesis H is true and sampling uncertainty (chance) is the reason

for the observed result.

IIThe hypothesis H is false and the difference between the true situation and H is the reason for the observed result.

The objective measure which is used to evaluate the degree to which the

data are consistent with the hypothesis H is a probability calculation of the sampling uncertainty referred to in Explanation I. If we assume H is true, we can calculate how likely, or unlikely, the outcome observed in the sample would be. If the outcome is likely (normally, this is thought to be the case when the significance level is at least 0.05, i.e., p 6 0.05), then we have meager evidence, at best, for concluding that the data are inconsistent with H. However, if the outcome is unlikely when H is true (normally, this is thought to be the case when the significance level is less than 0.05, i.e., p ! 0.05), then both I and II become plausible explanations. Provided the sampling procedure has been carefully designed to guard against gross defects, as the significance level (p- value) decreases we become less inclined to favor I, preferring II as the more plausible explanation for the apparent inconsistency between the data and the specific hypothesis H.

In summary, a significance test assumes that a specific hypothesis, H, about a population is true and compares the outcome observed in the sample with all other possible outcomes which sampling uncertainty might have generated. To carry out a test of significance, then, the following are necessary:

(1) A hypothesis about the population, usually referred to as H0 (the null hypothesis). In the example discussed in §2.2, H0 was the assumption that the population relapse rates, r1 and r2, were equal.

(2)Data from the population. These are obtained from a random sample and are usually summarized in the form of the observed value of a suitable test statistic. In the sample obtained by Thomas et al. [1] the test statistic was the number of patients who had received inadequate radiotherapy and who had subsequently relapsed in the operative site; its value in the sample was 2.

(3)A set of comparable events of which the outcome observed in the sample is only one possibility. In the example of §2.2, the five possible 2 ! 2 tables displayed in figure 2.1 constitute the set of comparable events.

(4)The probability distribution of the test statistic (see 2), based on the

assumption that the null hypothesis, H0, is true. For the example we discussed in §2.2, this probability distribution is displayed in table 2.3.

(5)A ranking of all possible outcomes in the set of comparable events (see

3)according to their consistency with the null hypothesis H0. In the example from Thomas et al. we ranked tables 0 through 4 as follows:

This ranking was based on the probability distribution displayed in table 2.3.

(6) A calculation of the probability that sampling uncertainty (chance) would produce an outcome no more consistent with the null hypothesis, H0, than the outcome which actually was observed (see 2). This probability is called the significance level of the data with respect to H0. In the example of §2.2, the significance level of the data with respect to the hypothesis of equal relapse rates was 0.041, the probability of obtaining tables 2, 3 or 4.

Many tests of significance for standard situations have been devised by statisticians, and the items outlined in (1)–(6) above have been specified. Often, all that a researcher must do in a particular circumstance is to select an appropriate test, evaluate the prescribed test statistic using the collected data, calculate the significance level (p-value) of the data with respect to the null hypothesis, H0, and draw an appropriate conclusion.

In succeeding chapters, we will describe particular tests of significance which are basic to medical statistics. In general, these chapters adopt a more relaxed presentation of specific significance tests than we have undertaken in this chapter. The particular test which we discussed in §2.2 is known as Fisher’s test for 2 ! 2 contingency tables. This test is sufficiently useful to justify a presentation in more general terms. In addition, since the specific details of the example in §2.2 are now familiar ground, a discussion of Fisher’s test should ease the transition of any hesitant reader onto the new aspects of well-known territory which we shall introduce in chapter 3.

Table 3.1. A 2 ! 2 table summarizing binary data collected from two groups

failure; these will be numbers between 0 and 1 which have the property that their sum is always 1, since success and failure are the only possible outcomes. We could describe each rate or proportion equally well as the probability of the corresponding outcome, whether success or failure. Then if p represents the probability of success, the probability of failure will be 1 – p since the probabilities of all possible outcomes in a population always sum to 1.

In the basic model for the population, we assume that each individual has a probability of success which can be specified by a value of p between 0 and 1. Therefore, the aim of the study which is summarized in a 2 ! 2 contingency table might be to compare two population groups, 1 and 2 say, with respect to their probabilities of success. To devise a test of significance suited to this purpose we must also assume:

(a)The probabilities of success for each member of Groups 1 and 2 are p1 and p2, respectively; i.e., within each group, the probability of success does not vary from individual to individual. Random sampling of individuals from a well-defined population of interest makes this assumption a reasonable one to adopt.

(b)For any member of either group, the outcome which occurs (success/ failure) does not influence the outcome for any other individual.

Once the data for the study have been collected (number of successes, failures in each group), they can be summarized in a 2 ! 2 contingency table such as the one shown in table 3.1.

The symbols R1 and R2 represent the total numbers of observations from Groups 1 and 2, respectively; the letter R is used because these are row totals

for the table. Likewise, C1 and C2 represent the column totals for the categories Success and Failure. The total number of observations in the sample is N, and

clearly N = R1 + R2 = C1 + C2. We will discuss the choice of letters appearing in the four cells of the 2 ! 2 table shortly. First, however, we need to specify the null hypothesis for Fisher’s test.

As we indicated above, it is quite natural in this particular situation to compare the probabilities of success in Groups 1 and 2. Therefore, the null hy-