Ординатура / Офтальмология / Английские материалы / Study Design and Statistical Analysis a practical guide for clinicians_Katz _2006

Definition

The degrees of freedom are the number of independent units of information used to calculate a particular statistic.

the null hypothesis were true than we would expect that diabetics would represent 14.72% of the deaths and 14.72% of the persons still alive. Similarly, we would expect that persons without diabetes would represent 85.28% of the deaths and 85.28% of the non-deaths.

To determine the number of subjects in each cell perform multiplication as

No death among persons without diabetes 0.8528 4727 4031

We can now fill in the two-by-two table (Table 5.4) assuming the null hypothesis is true.

In Table 5.5, I have placed the actual data on the relationship between diabetes and death.

Comparing Tables 5.4 and 5.5 you can see that the actual results differ substantially from what was expected if the null hypothesis were true. Forty-seven

Definition

The degrees of freedom are the number of independent units of information used to calculate a statistic.

The chi-squared test is invalid when the expected number of subjects per cell is 5.

diabetics died although we anticipated only 30 would have died. Among persons without diabetes 679 died but we had anticipated 696 would have died. This is why the chi-squared is large.

To determine the P-value from the chi-squared test you have to know the degrees of freedom. The degrees of freedom are the number of independent units of information used to calculate a particular statistic.

Although this may sound complicated, Table 5.6 illustrates how easy this is to determine for a two-by-two table. I have filled in cell “a”. From this one cell there is only one way you can fill in the other three cells of the table (e.g., b 726 47 679, d 4727 679 4048, etc.). This means that a two-by- two table has only one degree of freedom.

Knowing the chi-squared value and the degrees of freedom you can determine the P-value from the tables that used to be at the back of every statistic book. In this computer age, it is rare to look up the probability of a test result using a table. This is done by the computer (and therefore I have not placed any statistical tables at the back of this book!). You will see the number of degrees of freedom often printed out next to your analysis; it is helpful to know what it means.

To obtain a valid chi-squared test, the expected number of subjects per cell must be at least 5. I have italicized the word expected to remind you that it is not the observed number of subjects per cell that determines whether the chi-squared is valid. This means that with small sample sizes (e.g., 50) you need to determine the expected number of subjects per cell before using the chi-squared test. This would be tedious to do by hand. Fortunately, most computer programs will tell you automatically if the expected number of subjects is 5 in any cell of the table.

For example, Villar and colleagues investigated the cause of a botulism outbreak among bus drivers in Argentina.60 One of the foods they investigated is matambre, a traditional meat dish of Argentina that is cooked at temperatures too low to kill Clostridium botulinum spores.

As with the diabetes example, let us start with the marginal totals (Table 5.7). To determine the number of subjects expected in each cell, we fill in the cells

of the table assuming that the null hypothesis is true.

If eating matambre were not associated with botulism we would expect that bus drivers who ate matambre would represent an equal proportion of cases and non-cases of botulism. We know from the marginal totals in Table 5.7 that 52% of the drivers ate matambre and 48% did not. Therefore, if the null hypothesis were true than we would expect that 52% of the botulism cases and 52% of the non-cases ate matambre.

Tip

Use Fisher’s exact test when the expected cell size is 5.

To determine the expected number of subjects in each cell perform multipli-

No botulism among non-eaters of matambre 0.48 12 6

As the bolded cell is expected to have fewer than 5 subjects it would not be valid to use the chi-squared test. Instead, use Fisher’s exact test to assess the association between two dichotomous variables when the expected cell frequency is 5 subjects. It is never wrong to use the Fisher’s exact test instead of chi-squared. The major reason we traditionally used chi-squared test, where applicable, rather than the Fisher’s exact test is that the latter is computationally much more difficult. However, with the increased speed of modern computers this has become much less of an issue.

Fisher’s exact test determines the probability of obtaining a particular pattern of data given all possible arrangements of the observations. Fisher’s exact test can be computed assuming one or two tails; you will almost always want to use the two-tailed test (Section 2.8).

Table 5.8 shows the actual data on the association between consumption of matambre and botulism. You can see that there is a very strong relationship between eating matambre and developing botulism. The probability of getting such an association by chance is very small, reflected in the significant P-value for the Fisher’s exact test.

A limitation of both the chi-squared and Fisher’s exact test is that they do not measure the strength of the association between the risk factor and the outcome. (If you were thinking that a small P-value told you that it was a strong relationship remember the example of the tossed coin (Section 1.1)). P-values only tell you the probability that the observed association could have occurred by chance if there were no true association between eating matambre and botulism. With large sample sizes even small differences may be statistically significant.

Definition

The risk ratio is the ratio of the probability of occurrence in one group to the probability of occurrence in the other group.

Values are represented as n (%).

Fisher’s exact test (two-tailed) 0.0002.

Data from Villar, R.G., et al. Outbreak of Type A botulism and development of a botulism surveillance and antitoxin release system in Argentina. J. Am. Med. Assoc. 1999; 281: 1334–8, 1340.

Two commonly used tests to show the strength of an association are the risk ratio and the odds ratio. Both tell you how much more likely the outcome is to occur if the risk factor is present.

The risk ratio61 is the probability of the outcome occurring in one group (e.g., treatment group) divided by the probability of the outcome occurring in the other group (e.g., placebo group).

Looking at our two-by-two table, the probability of an event in the group for which the risk factor is present is a/(a b) and the probability of an event in the group for which the risk factor is absent is c/(c d). Therefore, the risk ratio equals:

risk ratio a/(a b) c/(c d)

The risk ratio tells you how much more likely the outcome is to occur if the risk factor is present than if the risk factor is absent.

Let us compute the risk ratio of death due to diabetes from the data shown in Table 5.5.

This means that death is about 1.71 times more likely to occur among persons with diabetes than among persons without diabetes.

A risk ratio of 1.0 means that the outcome is less likely to occur if the risk factor is present. For example, if the risk ratio of death were 0.5 in persons who

61In some books the risk ratio will be referred to as the relative risk. It is best to think of the relative risk as a family of measures for comparing two groups. The risk ratio, the rate ratio, the prevalence ratio, and the hazard ratio are all forms of the relative risk. In general, it’s best to use the more specific term.

When the 95% confidence intervals of the risk ratio exclude 1, we say that there is a statistically significant increase (or decrease) in the risk of the outcome.

Which risk ratio (1.71 or 0.58) correctly expresses the association between diabetes and death? They both do. Saying that death is 1.71 times more likely among diabetics than persons without diabetes is mathematically equivalent to saying that death is 0.58 times less likely among persons without diabetes than diabetics. To prove that to yourself take the reciprocal of 1.71:

1

0.58

1.71

Risk ratios should always be reported with confidence intervals. By convention, if the 95% confidence intervals exclude 1, then we say that there is a statistically significant (at P 0.05) increase (or decrease) in the risk of the outcome. When a higher degree of precision is needed you may report the 99% confidence intervals, and for exploratory studies you may want to report the 90% confidence intervals.

As the risk ratio is based on comparing the probabilities of an outcome (with and without the risk factor) it can also be used to calculate associations in crosssectional studies. Although the formula for risk ratio is the same whether it is

Risk ratio cannot be used with case–control studies.

Definition

The odds ratio is the ratio of the odds of disease among those with the risk factor to the odds of disease among those without the risk factor.

calculated for a prospective or cross-sectional study, when it is based on a crosssectional study it should be referred to as the prevalence ratio. For example, Ebrahim and colleagues found that the prevalence of smoking was 11.8% among pregnant women and 23.6% among non-pregnant women. Therefore, the prevalence ratio is 0.5 (11.8%/23.6%).62

The risk ratio cannot, however, be used with case–control studies. The reason is that it is meaningless to speak of the probability of an outcome occurring in a case–control study. The probability of an outcome among the cases is 100% (that’s what makes them cases) and the probability of an outcome among the controls is 0% (that’s what makes them controls.) The probability of an outcome occurring in the entire sample (cases and controls) is determined by the investigator! If the investigator chooses one control per case then the probability of outcome in the sample will be 50%; if the investigator chooses three controls per case then the probability of outcome in the sample will be 25%, etc.

Instead, with case–control studies we use the odds ratio (OR). The odds ratio is the ratio of the odds of disease among those with the risk factor to the odds of disease among those without the risk factor.

Looking back at Table 5.2, the odds of disease among those with the risk factor is a/b and the odds of disease among those without the risk factor is c/d. So the ratio is:

a/b

c/d

This can be rearranged to:

a d a d b c b c

As with risk ratios, the odds ratio should always be reported with confidence intervals.

A useful property of the odds ratio is that when an outcome is uncommon ( 10–15%) the odds ratio approximates the risk ratio. For example, if we go back to the prospective study of diabetes as a risk factor for death (Table 5.5), the odds ratio would be:

Note that the odds ratio (1.77) is almost identical to the risk ratio (1.71)!

Tip

To calculate the odds ratio when you have cells with no subjects in them, add 1/2 to each cell.

When the outcome is common in either group, the odds ratio no longer approximates the risk ratio. The difference between these two interpretations of the odds ratio can be seen in a prospective study of patients with stroke all of whom received thrombolytic therapy.63 The investigators evaluated the impact of having cortical involvement on failure to improve at 24 hour.

Failure to improve was more common among those with cortical involvement (50.5%) than those without cortical involvement (16.0%) (Table 5.10). This is reflected in the risk ratio and odds ratio both being 1.0.

However, the sample was about evenly split between those who failed to improve (52%) and those who improved (48%). Since the outcome was common,64 the odds ratio (2.7) is substantially higher than the risk ratio (1.6).

Investigators often report the odds ratio rather than the risk ratio when the frequency of the outcome is 10–15%. The reason is that many studies, including this one on the effect of cortical involvement on clinical improvement, perform multiple logistic regression, which produces odds ratios not risk ratios. In this study, the investigators performed multiple logistic regression so as to statistically adjust the odds ratios for age, sex, and stroke severity. The adjusted odds ratio was essentially the same as the unadjusted value (OR 2.7; 95% CI 1.4–5.2). Although the odds ratio does not approximate the risk ratio when the frequency is 10–15%, the odds ratio is still a valid measure of the association between a risk factor and an outcome.

63Saposnik, G., Young, B., Silver, B., et al. Lack of improvement in patients with acute stroke after treatment with thrombolytic therapy. J. Am. Med. Assoc. 2004; 292: 1839–44.

64In determining whether the outcome occurs in 10–15% of the sample, use the less common state. In this study, the less common state is improvement (48%).

78

Tip

Be wary of pairwise comparisons if the overall chi-squared is not statistically significant.

Bivariate statistics

Table 5.12. Association between African-American ethnicity and poor glycemic control

Values are represented as n (%).

2 1.8; P 0.18.

from Table 5.11 whether poor glycemic control is significantly more common among African-Americans than among persons of other ethnicities.

To determine if poor glycemic control is significantly more common among African-Americans you would need to collapse Table 5.11 into a two-by-two table comparing African-Americans to persons of all other ethnicities (Table 5.12).

As indicated by the large chi-squared and the small P-value, poor glycemic control is more common among African-Americans than among non-African- Americans. But is poor glycemic control significantly more common among African-Americans than Latinos? You cannot answer this from either Tables 5.11 or 5.12. To answer this you would need to directly compare these two groups as shown in Table 5.13.

As you can see the chi-squared value is small and the P-value is 0.05. The difference in glycemic control between African-Americans and Latinos is not statistically significant.

In making pairwise comparisons such as those shown in Table 5.13, it is important to avoid capitalizing on chance. Specifically, it stands to reason that if you have four groups and you compare the highest group to the lowest group you are more likely to find a statistical difference than if you compare the four groups to each other. For this reason, if the overall chi-squared is not significant, pairwise