Ординатура / Офтальмология / Английские материалы / Principles Of Medical Statistics_Feinstein_2002

8

Confidence Intervals and Stability:

Binary Proportions

CONTENTS

8.1Sources of Complexity

8.2Jackknife Screening

8.3Gaussian Confidence Intervals

8.3.1Standard Error of a Proportion

8.3.2Confidence Interval for a Proportion

8.4Binomial (Bernouilli) Confidence Intervals

8.4.1Illustration of Bootstrap Procedure

8.4.2Bernouilli Procedure

8.5Mental and “No Events” Short-Cut Approximations

8.5.1Mental Approximation

8.5.2“Rumke’s Rule” for “No Events”

8.6“Continuity” and Other Corrections for Mathematical Formulas

8.6.1Customary “Continuity Correction”

8.6.2Other Corrections and Appraisals

8.7Interpretation of Stability

8.7.1Impact of Directional Variations

8.7.2Impact on Small Proportions

8.7.3Effect of Small Numerators

8.7.4Events-per-Variable Ratio

8.8Calculations of Sample Size

8.8.1Political Polls

8.8.2General Formula

8.8.3Sample Sizes in Medical Research

8.8.4Sample Size for Dimensional Data

8.8.5Checks on Calculations

8.8.6Enlargement for “Insurance”

8.9Caveats and Precautions

References

Exercises

The evaluation of stability and confidence intervals for binary proportions needs a separate chapter, because the methods will often differ from those discussed in Chapter 7 for dimensional means. The differences in basic structure of the binary data will make the methods sometimes similar, but other times either simpler or more complex.

Any proportion always has a binary structure, but the entities called binary proportions are the central indexes for a group of binary data, in which each item has one of two complementary categories, such as yes/no, failure/success, dead/alive. With the items coded as either 0 or 1, the array of data will be {0, 1, ..., 0, 1, 0, ..., 1}. If the group contains n items, r will be coded as 1, and n − r as 0. The central index can be chosen as either p = r/n or q = 1 − p = (n − r)/n. Thus, if 15 people are alive

© 2002 by Chapman & Hall/CRC

and 27 are dead, the group’s binary proportion can be the survival rate, 15/42 = .36, or the fatality rate, 27/42 = .64.

The binary proportions that commonly appear in medical statistics can describe diverse clinical entities, but can also refer to population “rates” (over intervals of time) for such demographic events as births, marriages, and deaths. The differences between proportions and rates are discussed in Chapter 17, but in pragmatic reporting, the two terms are often used interchangeably, and are also often cited with a third term, percentages. In the statistics of daily life, binary percentages can express baseball batting averages, weather forecasts, polls of public opinion, interest on bank accounts, discounts for sale merchandise, the composition of foods, and the concentration of atmospheric pollutants.

8.1Sources of Complexity

Although stability and confidence zones can sometimes be evaluated with the same theoretical and empirical procedures used for a mean, the rearrangements of binary proportions have three important features that increase the complexity.

1.Because each proportion can be cited as either p or q, rather than with a unique single value, the proportionate changes and potential variation of a rearrangement can differ dramatically if referred to p or q. Thus, a 6% increment in rates will seem relatively large if divided by the observed success rate of 10%, but much smaller if divided by the corresponding failure rate of 90%.

2.The language and concepts can become tricky when an increment such as .06 or 6% in two

proportions is divided by another proportion — such as the observed success rate of .10 or 10% — to form a third proportion, which is .06/.10 = .6 or 60%. If the published results

mention an “improvement of 60%,” you will not know whether the writers have cited a proportionate change (in which the compared values were 10% and 16%), or a direct incremental change in the observed success rate (which would have had to go from 10% to 70%). Furthermore, if someone wants to minimize rather than maximize the apparent accomplishment of a 6% increment between success rates of 10% and 16%, the change could be referred

to the original failure rate of 90%. With the latter citation, the proportionate decline in the failure rate would be .06/.90 = .067 or 7%. Thus, the same observed change could be reported

as 6%, 60%, or 7%, according to the choice made by the “seller.” All three citations are legitimate and accurate, but the “buyer” will have to beware.

3.Stability and confidence intervals for binary proportions can often be well estimated with the Gaussian mathematics used for means. If the group size is small, however, or if the proportion is far below (or above) 50%, the Gaussian procedure may yield distorted results. A more accurate set of calculations can be obtained with binomial (or Bernouilli) distributions, discussed in Section 8.4, for the 0/1 structure of the data.

8.2Jackknife Screening

The jackknife technique has already been illustrated at the beginning of Chapter 7 for examining stability in such proportions as 1/2, 1/3, 150/300, and 100/300.

A proportion constructed as r/n can change either to the smaller value of (r − 1)/(n – 1) if the removed member is a 1, or to the larger value of r/(n − 1) if the removed member is a 0. For example, if p = 2/7 = .2857, the reduced proportions will be either 1/6 = .1667 or 2/6 = .3333. The chance of getting the smaller value will be r/n, and the chance of getting the larger value will be (n − r)/n.

The strategy of examining the two extreme jackknife results (and the chances of getting those results) offers a quick, almost purely mental method of screening for the stability of a proportion. Regardless of how the potential values and chances are interpreted, the results here show that the 2/7 proportion is

© 2002 by Chapman & Hall/CRC

certainly unstable. Aside from a jackknife screening that shows obvious fragility, however, the stability of a proportion is usually evaluated with more formal mathematical methods.

8.3Gaussian Confidence Intervals

The same parametric reasoning used in Chapter 7 can be applied to estimate the standard error of a proportion and to determine confidence intervals. We assume that the parent population contains binary data, with π as the parametric proportion. When repeated samples of size n are drawn from the parent population in the theoretical sampling process, each sample will have pj as its central index. We can find the standard error of those sampled proportions by determing the average or “standard” deviation of pj − π . We then use the standard error to calculate a confidence interval for the zone in which π is located.

8.3.1Standard Error of a Proportion

The parametric strategy for determining standard error of a proportion is exactly analogous to what was done with a mean. You may recall, from Section 4.9.3, that the variance of an observed set of binary data was s2 = pq, which will estimate the parametric variance as σˆ 2 = pq. The standard error of the proportion will then be estimated as

This simple formula is possible because npq, the group variance of the binary data, was divided by n to form the variance in Section 4.9.3. Now that the degrees-of-freedom concept has been discussed, the reason for this divisor can be explained. When group variance was calculated for dimensional data as Σ X2i – nX2 , there were n degrees of freedom in the choice of the Xi values, and 1 d.f. in the choice of X . Thus, the value for d.f. was n − 1. When a binary proportion is determined as p, there are n degrees of freedom in choice of the 0 or 1 component values. As soon as p is known, however, the value of q is promptly found as q = 1 − p, and the group variance is also promptly found as npq, without any subtractions to reduce the n degrees of freedom. Accordingly, the group variance npq is divided by n to form the variance. The standard deviation, pq , is then divided by n to determine the standard error.

For example, if a political candidate is favored by 60% of voters in a poll, the standard error of the

8.3.2Confidence Interval for a Proportion

Another splendid attribute of Gaussian mathematics and binary proportions is that the values of Zj calculated for pj in repeated samplings will have a Gaussian distribution (if n is suitably large). With this principle, we can form the ratio

and interpret it as though it were a standard Gaussian deviate.

© 2002 by Chapman & Hall/CRC

Using the observed value of p for pj, and pq for σ , we can estimate boundaries for the location of the parametric π by choosing an appropriate value of Zα for the desired 1 – α level of the Gaussian confidence interval. After Formula [8.1] receives suitable substitution and conversion, the calculation becomes

and is an exact counterpart of the confidence interval Formula [7.3] for a mean. To compare the observed p with a hypothesized π , Formula [8.1] becomes the critical ratio

8.3.2.1 Example of Calculation — Suppose the observed proportion is 26/42 = .62. The value of the standard error pq ⁄n = (26 ⁄42)(16 ⁄42) ⁄42 = .00561 = .075. [With the short-cut formula, this calculation is (26 )(16 ) ⁄423 = .00561 = .075.] The 95% confidence interval will be

π = .62 ± (1.96)(.075) = .62 ± .147

and will extend from .473 to .767.

Suppose someone believes that this result, despite a group size of 42, is a “short-term fluke” and that the “long-run” parametric value in the population is actually π = .50. With Formula [8.3], we can calculate Z = (.62 − .50)/.075 = 1.6, for which 2P > .05. As discussed later, we could have anticipated that this result would have 2P > .05, because the 95% confidence interval included the parametric value of π = .50.

8.3.2.2 Example of Practical Application — Confidence intervals for proportions are constantly used on election night when the media make forecasts about a winner. A particular voting precinct (or cluster of precincts) is chosen as a random (or at least trustworthy) representative of the political region under scrutiny. As increasing numbers of votes accrue from the selected precinct(s), confidence intervals are repeatedly calculated. When a high-level confidence interval excludes .50, a victor will be predicted.

When the forecasts go wrong, the error may have scientific or statistical sources. Scientifically, the selected precincts may have been poorly chosen. Because of population migration or some other reason, the precincts may no longer suitably represent the entire region. If the precincts have been well chosen, however, the source of wrong predictions can be two statistical factors: the value of Zα chosen for the level of confidence, and the increment of tolerance by which the lower level of the estimate must exceed

.50. With lower values of confidence level and higher tolerance, the forecasts can be made earlier on election night (i.e., with smaller numbers of precinct ballots), but will have a greater margin for error.

For example, suppose the early results show Candidate A ahead of Candidate B by 60 votes to 40. With a 90% confidence interval, the estimate is

π= (60/100) ± 1.645 (.60 )(.40 ) ⁄100

=.60 ± (1.645)(.049) = .60 ± .08.

The confidence interval extends from .52 to .68. Since the “tie” value of .50 is excluded, Candidate A might be proclaimed the winner. On the other hand, with a 99% confidence interval, Zα will be 2.58. The interval will be .60 ± (2.58)(.049) = .60 ± .126 and extends from .484 to .726. With this result, Candidate A cannot yet be guaranteed victory. With Zα set at 1.96, a 95% confidence interval could be calculated as .6 ± (1.96)(.049) = .6 ± .096, and would extend from .504 to .696. If the forecasters are content to regard 50.4% as a suitable margin of tolerance above 50%, they might issue a victory prediction.

© 2002 by Chapman & Hall/CRC

8.3.2.3 Customary Applications and Abuses — In the foregoing discussion, we were trying to estimate a populational parameter from the results found in a sample; and a specific sampling process was either planned or had actually occurred. In most medical publications, however, the investigator has obtained results from an observed group of people (or other entities) who happened to be conveniently available for the research. For the binary proportion, p, that was found in these entities, the main goal is to determine the stability of the result, not to estimate a populational parameter.

Nevertheless, the goal of checking stability in convenient groups is almost always approached with the same strategy used for estimating parameters in random samples. A standard error and/or confidence interval, determined from the observed result, become used for appraising its stability. As the observed result is not a random sample, all of the mathematical reasoning based on randomness is not really appropriate or justified. To allow the reasoning to be used, however, the assumption is made that the observed group is indeed a random sample.

This abuse of the fundamental mathematics has persisted for about a century and has become sanctified by the recommendation and approbation received from authoritative statisticians and as well by constant appearance in medical publications. A major advantage of bootstrap (or Bernouilli) confidence intervals discussed in the next section is that they come only from the observed data, without making parametric estimates and without requiring mathematical assumptions about parametric sampling or distributional shapes.

8.4Binomial (Bernouilli) Confidence Intervals

A different way to determine standard errors and confidence intervals for binary proportions is both empirical and theoretical. Empirically, the observed group of n members is used directly as a parent population in a bootstrap resampling procedure. Theoretically, the results of the binary sampling process can be exactly anticipated with a mathematical model. Applied to items having the binary or “binomial” values of 0 or 1, the model is often called the binomial distribution.

8.4.1Illustration of Bootstrap Procedure

Consider the binary proportion 6/10 = .60. Because each of the ten people in this group can be rated as 0 or 1, success or failure, yes or no, the existing group of data can be depicted as a set of six members of one type and four of the other. One such depiction would be the data set {Y, Y, Y, Y, Y, Y, N, N, N, N}, where each Y = yes and each N = no.

8.4.1.1 Formation of Samples — To show each member in a complete bootstrap resampling would require 1010 samples. The process is therefore easier to demonstrate if we do the complete bootstrap for 5 items, which will involve only 105 = 100,000 samples from the set of data.

Each time an item is chosen, there are 10 possibilities. For the first choice in the sample, we could get any one of the 6 Y items, or any one of the 4 N items. For the second choice, we could again get any one of the 6 Y items or any one of the 4 N items. The 100 possibilities in the first two choices would thus contain the patterns

NN:4 × 4 = 16 possibilities

These four patterns contain three sets of contents: YY, NN, and a set that has one N and one Y.

With 10 more chances in the third choice, the total number of possibilities would be expanded to 1000, the number of patterns to eight, and the number of different contents to four. Among the four sets of contents, one group would have 3 Y’s; one would have2 Y’s and 1 N; one would have 1 Y and 2 N’s;

© 2002 by Chapman & Hall/CRC

and one would have 3 N’s. The fourth and fifth choices would expand the number of possibilities to 10,000 and then to 100,000; the number of patterns would go to 16 and then to 32; the number of sets with different contents would go to five and then to six.

8.4.1.2 Summary of Results — The sequence of possible patterns and contents for the foregoing choices is shown in Table 8.1.

The 100,000 possible patterns after five choices will be arranged in the six basic formats of: 5 Y’s; 4 Y’s and 1 N; 3 Y’s and 2 N’s; 2 Y’s and 3 N’s; 1 Y and 4 N’s; and 5 Y’s. Table 8.2 shows a summary of the patterns, frequencies, and relative frequencies of the contents in Table 8.1. For each set, the respective proportions, pi, are 1.00, .80, .60, .40, .20, or 0. The relative frequency, fi, of choices for each proportion, pi, is shown in the sixth column of Table 8.2. [The far right column, binomial relative frequency, is explained in Section 8.4.2.3.]

8.4.1.3 Mean, Variance, and Standard Error — To find the mean value for the proportion of Y’s in the 100,000 bootstrapped samples of Table 8.2, we can multiply each value of pi by its total number of choices, mi, and then divide by 100,000. Alternatively, we can multiply each pi by its relative frequency fi and then take the sum of the products as

Σ pi fi

With the latter calculation, the bootstrap-sample mean is (1.00)(.0778) + (.80)(.2592) + (.60)(.3456) + (.40)(.2304) + (.20)(.0768) + (0)(.0102) = .60, which is the expected result for p.

For the variance of the bootstrapped samples, we can use the formula

Σ fi(pi – p)2

which yields (.0778)(1.00 − .60)2 + .2592(.80 − .60)2 + .3456(.60 − .60)2 + .2304(.40 − 60)2 + .0768

8.4.1.4 Confidence Intervals — The main virtue of Tables 8.1 and 8.2 is that they show the exact distribution of proportions in the array of bootstrap samples. An important thing to note about this distribution is that the relative frequencies are not arranged symmetrically around the central index value of p = .60. This asymmetry will occur whenever the sample sizes are relatively small (i.e., n < 20 or 30) for any proportion in which p is not at the “meridian” value of .50. Consequently, the symmetrical confidence interval calculated with the component value, ± Zα pq ⁄n , will not be wholly accurate with small sample sizes. Thus, just as the t distribution is best used for small samples of dimensional data, the Bernouilli binomial distribution (which is discussed shortly, and which provides the same results as the bootstrap) is best used for small samples of binary data.

From Table 8.2, we can demarcate a symmetrical confidence interval using the proportions .40 to .80. They would include .2304 + .3456 + .2592 = .8352 or 84% of the data. The proportions from .20 to 1.00 would create an asymmetrical interval that contains 99% of the data, with the 0 value of the proportion excluded.

As noted shortly, the Bernouilli or binomial distribution takes a Gaussian shape when n is large. (The relative frequencies in Table 8.2 already look somewhat Gaussian despite the small n of 5.) Accordingly, the Gaussian Z indexes are often applied to estimate confidence intervals for a proportion, using the

of .40 to .80 found with the bootstrap.

© 2002 by Chapman & Hall/CRC

TABLE 8.1

100,000 Possible Patterns in Five Selections from Population with Binary Proportion of .6

© 2002 by Chapman & Hall/CRC

TABLE 8.2

Summary of Results in Table 8.1 for p = .6, q = .4

8.4.2Bernouilli Procedure

The diverse possibilities in a bootstrap sampling of binary data were mathematically determined by Jacob Bernouilli more than three centuries ago. He formulated an elegant expression, called the binomial expansion, which is

(p + q) n

You met the simple forms of this expansion in your elementary algebra as (a + b)2 = a2 + 2ab + b2, and (a + b)3 = a3 + 3a2b + 3ab2 + b3.

To illustrate how the expansion works, suppose we toss a coin three times. With H for head and T for tails, there are 8 possible outcomes: HHH, HHT, HTH, HTT, THH, THT, TTH, and TTT. One of these eight outcomes has all heads; three have two heads and one tail; three have one head and two tails; and one has all tails. This result could have been expected from the binomial expansion of (p + q)3, if we let p = probability of a head and q = probability of a tail. The expansion is p3 + 3p2q + 3pq2 + q3. The probability of getting all heads is p3 = (1/2)3 = 1/8. The probability of getting two heads and a tail is 3p2q = 3/8. Correspondingly, 3pq2 = 3/8 and q3 = 1/8 are the probabilities for the other two outcomes.

8.4.2.1General Format of Binomial Terms — In general application, the binomial expansion for a sample of n items has n + l terms. They are pn, pn−lq, pn−2q2, … , pn−rqr, … , p2qn−2, pqn−1, qn.

The values of r increase successively from 0 for the first term to n for the last. Each term will reflect a sampled proportion having the value (n − r)/n. For example, in Table 8.2, the sampled proportion associated with p5 was 5/5 = l; the sampled proportion associated with p2q3 was 2/5 = .40.

8.4.2.2Binomial Coefficients — Each term will also have an associated binomial coefficient,

The binomial coefficient is particularly worth knowing about because it also plays an important role later in Chapter 12 when we meet the Fisher Exact Probability test for comparing two proportions. The coefficient represents the different number of individual combinations that can be formed when n things are taken r at a time. The first term can be chosen in n different ways. The next term can then be chosen from n − 1 items. The third term can come from n − 2 items, and so on. Thus, there are n × (n − 1) × (n − 2) × … × 2 × 1 = n! sequences, called permutations, in which the n items could be arranged. If only r items are selected from the total group of n, the permutation will extend from n down to n − r + 1.

© 2002 by Chapman & Hall/CRC

For example, suppose you are being dealt 13 consecutive cards from a well-shuffled deck of 52. The first card can be chosen in 52 ways, the second in 51, the third in 50, and so on down to the 13th in 40 ways. The value of 40 is n − r + 1 = 52 − 13 + 1. Accordingly, the sequence in which you get the cards

possibilities from the original 52!. The general formula for the sequence of r things taken from a group of n is n!/(n − r)!.

In receiving the cards, however, you do not care about the sequence in which they arrive. What concerns you is the particular combination of 13 cards that constitutes your “hand.” Because those 13 cards could themselves be permuted in 13! sequences, the combination in your particular hand could have been constructed in (52!/39!) ÷ 13! ways. The latter expr ession is 52!/[(39!)(13!)]. If you work out this calculation, the result turns out to be 8.0658 × 10 67/[(2.0398 × 10 46 )(6.227 × 10 9)] = 6.3501 × 10 11.

The general formula for the combination that forms your “hand” is therefore n!/[(n – r)! (r!)]. An important thing to know when working with factorials is that 0! = 1. If you take all n things at once, there is only one possible combination. It is n!/[(n!)(0!)] = 1.

8.4.2.3 Application of Binomial Coefficients and Terms — The binomial expansion of (p + q)n can be expressed as

The first term, with r = 0, is {n!/[(n!)(0!)]}pn = pn. The second term, with r = 1, is {n!/[(n – 1)! (1!)] }pn− 1q = npn−1q. The third term with r = 2, is {n!/[(n − 2)!(2!)]}pn−2q2 = [n(n − 1)/2]pn–2q2, and so on down to qn.

For example, the binomial expansion when n = 5 is p5 + 5p4q + 10p3q2 + 10p2q3 + 5pq4 + q5. The two coefficients of 5 came from 5!/(4!1!) and the two coefficients of 10 came from 5!/(3!2!). Each of these terms will indicate the relative frequency, i.e., the probability, of getting the particular arrangement of 1’s, and 0’s implied by the pn−rqr part of the term. Thus, when n = 5 and r = 2, there would be three 1’s and two 0’s, producing the proportion 3/5 or (n − r)/n. The chance of occurrence for this proportion would be 10p3q2 = 10(3/5)3 (2/5)2 = .346. Each of the nr pn−rqr terms represents the binomial relative frequency, cited in the far right column of Table 8.2, for the pi proportion that is calculated as (n − r)/n.

8.4.2.4 Example of Application — In Table 8.2, the bootstrap sampling showed the possibilities occurring when samples of 5 members were taken from a population with π estimat ed to be .6. If we apply the Bernouilli process with n = 5, p = .6, and q = 1 − .6 = .4, we get the binomial relative frequencies shown in the far right column of Table 8.2. These relative frequencies are identical to those obtained previously as values of fi with the bootstrap process.

As another example, suppose we want to evaluate the proportion of spades that would be obtained in five selections, each from a full deck of cards. Because p = 1/4, we obtain the possibilities are shown in Table 8.3.

TABLE 8.3

Bernouilli Distribution or Probability for Spades Found in 5 Selections Each from a Full Deck of Cards

© 2002 by Chapman & Hall/CRC

In this instance, proportions of spades ranging from 0 to .6 would be found in .2373 + .3955 + .2637 +

.0879 = .9844 of the selections. We could thus form a nonsymmetrical 98% confidence interval, demarcated by values from 0 to .6, for the expected proportions if five items were taken from a distribution with π = .25. Note that the interval of 0 to .6 is not symmetrical around the value of .25. Note also that the expected proportion of .25 was impossible to obtain in a sample of 5 members. The highest individual probabilities were .3955 for getting a proportion of .2 for the spades, and .2637 for a proportion of .4.

If we had tried to get this result with a Gaussian calculation, the closest approach would be with Z.01 = 2.58 for a 99% confidence level. The standard error would be (1 ⁄4)(3 ⁄4 ) ⁄5 = .194 and the interval would be .25 ± (2.58)(.194) = .25 ± .50. It would extend from an impossible −.25 to .75. (In the binomial expansion, the interval from 0 to .8 would have covered 99.9% of the possibilities.)

8.4.2.5 Binomial Expansions for Confidence Intervals — Because of the possible asymmetries and unrealistic results, the confidence intervals for proportions are best determined with binomial expansions when the sample or group size, n, is small. As n enlarges, the binomial and Gaussian results become less disparate.

For example, suppose the proportion of .2 has been observed as 4 successes and 16 failures in a group of 20. The binomial expansion would contain 21 terms for possible proportions whose values would be 1, .95, .90, ..., .10, .05, 0. The binomial coefficients and terms would produce tiny values for the relative frequencies of the first ten terms with potential proportions of 1, .95, ... , .60, .55. For the remaining 11 terms, shown in Table 8.4, about 84% (= .1090 + .1745 + ... + .1369) of the possible proportions would be in the zone from .10 to .30. An additional 11% (= .0545 + .0576) would be included if the zone extends from .05 to .35. Thus, the interval from .05 to .35 would include 95% of the possible proportions if a sample of 20 members was drawn from a binary population with parameter π = .20. The tactic of getting the confidence interval by using the exact binomial formula and working toward the center from both ends is sometimes called the Clopper-Pearson method.1

TABLE 8.4

Demonstration of Bernouilli Probabilities for 11 terms of p = 4/20 = .20

If this same 95% confidence interval were calculated with Gaussian arithmetic, the result would be

.20 ± (1.96) (.2 )(.8 ) ⁄20 = .20 ± .175. The interval would extend from .025 to .375. [If you wanted to use a t-distribution approach for the small sample, the 95% confidence interval for 19 degrees of freedom would be .20 ± (2.093) (.2 )(.8 ) ⁄19 = .20 ± .192. This interval would extend from .008 to .392.] Both of the Gaussian and t parametric calculations are now much closer to, but are still not quite the

same as, the “gold standard” zone of .05 to .35 obtained with the binomial Clopper-Pearson method.

8.4.2.6 Binomial Tactics for Rare Events — A particularly valuable use of the binomial calculations occurs when claims are made about rare or uncommon events. For example, suppose an

© 2002 by Chapman & Hall/CRC