An Introduction To Statistical Inference And Data Analysis

average of X1; : : : ; Xn. It will be helpful to identify several crucial pieces of information for each random variable of interest:

First we consider Xi. Notice that converting to standard units does not change the shape of the distribution of Xi. For example, if Xi » Bernoulli(0:5), then the distribution of Xi assigns equal probability to each of two values, x = 0 and x = 1. If we convert to standard units, then the

distribution of

Z1 = Xi ¡ ¹ = Xi ¡ 0:5 ¾ 0:5

also assigns equal probability to each of two values, z1 = ¡1 and z1 = 1. In particular, notice that converting Xi to standard units does not automatically result in a normally distributed random variable.

Next we consider the sum and the average of X1; : : : ; Xn. Notice that, after converting to standard units, these quantities are identical:

It is this new random variable on which we shall focus our attention. We begin by observing that

£p ¡ ¹ ¡ ¢¤ 2 2

Var n Xn ¹ = Var (¾Zn) = ¾ Var (Zn) = ¾

is constant. The WLLN states that

¡ ¹ ¢ P

Xn ¡ ¹ ! 0;

p

so n is a \magni¯cation factor" that maintainsprandom variables with a constant positive variance. We conclude that 1= n measures how rapidly the sample mean tends toward the population mean.

122 CHAPTER 6. SUMS AND AVERAGES OF RANDOM VARIABLES

Now we turn to the more re¯ned question of how the distribution of the sample mean changes as the sample mean tends toward the population mean. By converting to standard units, we are able to distinguish changes in the shape of the distribution from changes in its mean and variance. Despite our inability to make general statements about the behavior of Z1, it turns out that we can say quite a bit about the behavior of Zn as n becomes large. The following theorem is one of the most remarkable and useful results in all of mathematics. It is fundamental to the study of both probability and statistics.

Theorem 6.2 (Central Limit Theorem) Let X1; X2; : : : be any sequence of independent and identically distributed random variables having ¯nite mean ¹ and ¯nite variance ¾2. Let

¹ ¡

Zn = X¾=n pn¹;

let Fn denote the cdf of Zn, and let © denote the cdf of the standard normal distribution. Then, for any ¯xed value z 2 <,

P (Zn · z) = Fn(z) ! ©(z)

as n ! 1.

The Central Limit Theorem (CLT) states that the behavior of the average (or, equivalently, the sum) of a large number of independent and identically distributed random variables will resemble the behavior of a standard normal random variable. This is true regardless of the distribution of the random variables that are being averaged. Thus, the CLT allows us to approximate a variety of probabilities that otherwise would be intractable. Of course, we require some sense of how many random variables must be averaged in order for the normal approximation to be reasonably accurate. This does depend on the distribution of the random variables, but a popular rule of thumb is that the normal approximation can be used if n ¸ 30. Often, the normal approximation works quite well with even smaller n.

Example 1 A chemistry professor is attempting to determine the conformation of a certain molecule. To measure the distance between a pair of nearby hydrogen atoms, she uses NMR spectroscopy. She knows that this measurement procedure has an expected value equal to the actual distance and a standard deviation of 0:5 angstroms. If she replicates the experiment

The CLT has a long history. For the special case of Xi » Bernoulli(p), a version of the CLT was obtained by De Moivre in the 1730s. The ¯rst attempt at a more general CLT was made by Laplace in 1810, but de¯nitive results were not obtained until the second quarter of the 20th century. Theorem 6.2 is actually a very special case of far more general results established during that period. However, with one exception to which we now turn, it is su±ciently general for our purposes.

The astute reader may have noted that, in Examples 1 and 2, we assumed that the population mean ¹ was unknown but that the population variance ¾2 was known. Is this plausible? In Examples 1 and 2, it might be that the nature of the instrumentation is su±ciently well understood that the population variance may be considered known. In general, however, it seems somewhat implausible that we would know the population variance and not know the population mean.

The normal approximations employed in Examples 1 and 2 require knowledge of the population variance. If the variance is not known, then it must be estimated from the measured values. Chapters 7 and 8 will introduce procedures for doing so. In anticipation of those procedures, we state the following generalization of Theorem 6.2:

Theorem 6.3 Let X1; X2; : : : be any sequence of independent and identically distributed random variables having ¯nite mean ¹ and ¯nite variance ¾2. Suppose that D1; D2; : : : is a sequence of random variables with the prop-

erty that D2 P ¾2 and let

n !

¹ ¡

Tn = Xn p¹ : Dn= n

Let Fn denote the cdf of Tn, and let © denote the cdf of the standard normal distribution. Then, for any ¯xed value t 2 <,

P (Tn · t) = Fn(t) ! ©(t)

as n ! 1.

We conclude this section with a warning. Statisticians usually invoke the CLT in order to approximate the distribution of a sum or an average of random variables X1; : : : ; Xn that are observed in the course of an experiment. The Xi need not be normally distributed themselves|indeed, the grandeur of the CLT is that it does not assume normality of the Xi. Nevertheless, we will discover that many important statistical procedures

126 CHAPTER 6. SUMS AND AVERAGES OF RANDOM VARIABLES

do assume that the Xi are normally distributed. Researchers who hope to use these procedures naturally want to believe that their Xi are normally distributed. Often, they look to the CLT for reassurance. Many think that, if only they replicate their experiment enough times, then somehow their observations will be drawn from a normal distribution. This is absurd! Suppose that a fair coin is tossed once. Let X1 denote the number of Heads, so that X1 » Bernoulli(0:5). The Bernoulli distribution is not at all like a normal distribution. If we toss the coin one million times, then each Xi » Bernoulli(0:5). The Bernoulli distribution does not miraculously become a normal distribution. Remember,

The Central Limit Theorem does not say that a large sample was necessarily drawn from a normal distribution!

On some occasions, it is possible to invoke the CLT to anticipate that the random variable to be observed will behave like a normal random variable. This involves recognizing that the observed random variable is the sum or the average of lots of independent and identically distributed random variables that are not observed.

Example 3 To study the e®ect of an insect growth regulator (IGR) on termite appetite, an entomologist plans an experiment. Each replication of the experiment will involve placing 100 ravenous termites in a container with a dried block of wood. The block of wood will be weighed before the experiment begins and after a ¯xed number of days. The random variable of interest is the decrease in weight, the amount of wood consumed by the termites. Can we anticipate the distribution of this random variable?

The total amount of wood consumed is the sum of the amounts consumed by each termite. Assuming that the termites behave independently and identically, the CLT suggests that this sum should be approximately normally distributed.

When reasoning as in Example 3, one should construe the CLT as no more than suggestive. Most natural processes are far too complicated to be modelled so simplistically with any guarantee of accuracy. One should always examine the observed values to see if they are consistent with one's theorizing. The next chapter will introduce several techniques for doing precisely that.

6.3Exercises

1.Suppose that I toss a fair coin 100 times and observe 60 Heads. Now I decide to toss the same coin another 100 times. Does the Law of Averages imply that I should expect to observe another 40 Heads?

2.Chris owns a laser pointer that is powered by two AAAA batteries. A pair of batteries will power the pointer for an average of ¯ve hours use, with a standard deviation of 30 minutes. Chris decides to take advantage of a sale and buys 20 2-packs of AAAA batteries. What is the probability that he will get to use his laser pointer for at least 105 hours before he needs to buy more batteries?

3.A certain ¯nancial theory posits that daily °uctuations in stock prices are independent random variables. Suppose that the daily price °uctuations (in dollars) of a certain blue-chip stock are independent and

identically distributed random variables X1; X2; X3; : : :, with EXi = 0:01 and Var Xi = 0:01. (Thus, if today's price of this stock is $50, then tomorrow's price is $50 + X1, etc.) Suppose that the daily price °uctuations (in dollars) of a certain internet stock are independent and identically distributed random variables Y1; Y2; Y3; : : :, with EYj = 0 and Var Yj = 0:25.

Now suppose that both stocks are currently selling for $50 per share and you wish to invest $50 in one of these two stocks for a period of 50 market days. Assume that the costs of purchasing and selling a share of either stock are zero.

(a)Approximate the probability that you will make a pro¯t on your investment if you purchase a share of the blue-chip stock.

(b)Approximate the probability that you will make a pro¯t on your investment if you purchase a share of the internet stock.

(c)Approximate the probability that you will make a pro¯t of at least $20 if you purchase a share of the blue-chip stock.

(d)Approximate the probability that you will make a pro¯t of at least $20 if you purchase a share of the internet stock.

(e)Approximate the probability that, after 400 days, the price of the internet stock will exceed the price of the blue-chip stock.

128 CHAPTER 6. SUMS AND AVERAGES OF RANDOM VARIABLES