An Introduction To Statistical Inference And Data Analysis

There are ¡52¢ ways of choosing the two dice on which a 5 appears and 5 ¢ 5 ¢ 5 ways of choosing a di®erent number for each of the three remaining dice. Hence,

is the probability that at least two students share a common birthday?

As is inevitably the case with constructing mathematical models of actual phenomena, some simplifying assumptions are required to make this problem tractable. We begin by assuming that there are 365 possible birthdays, i.e. we ignore February 29. Then the sample space, S, of possible birthdays for k students comprises 365k outcomes.

Next we assume that each of the 365k outcomes is equally likely. This is not literally correct, as slightly more babies are born in some seasons than in others. Furthermore, if the class contains twins, then only certain pairs of birthdays are possible outcomes for those two students! In most situations, however, the assumption of equally likely outcomes is reasonably plausible.

Let A denote the event that at least two students in the class share a birthday. We might attempt to calculate

P (A) = #(#(AS));

but a moment's re°ection should convince the reader that counting the number of outcomes in A is an extremely di±cult undertaking. Instead, we invoke Theorem 2.1 and calculate

P (A) = 1 ¡ P (Ac) = 1 ¡ #(Ac): #(S)

This is considerably easier, because we count the number of outcomes in which each student has a di®erent birthday by observing that 365 possible birthdays are available for the oldest student, after which 364 possible birthdays remain for the next oldest student, after which 363 possible birthdays remain for the next, etc. The formula is

# (Ac) = 365 ¢ 364 ¢¢¢(366 ¡ k)

and so

P (A) = 1 ¡ 365 ¢ 364 ¢¢¢(366 ¡ k) : 365 ¢ 365 ¢¢¢365

The reader who computes P (A) for several choices of k may be astonished to discover that a class of just k = 23 students is required to obtain P (A) > :5!

2.4Conditional Probability

Consider a sample space with 10 equally likely outcomes, together with the events indicated in the Venn diagram that appears in Figure 2.4. Applying the methods of Section 2.3, we ¯nd that the (unconditional) probability of

A is

P (A) = #(A) = 3 = :3: #(S) 10

Suppose, however, that we know that we can restrict attention to the experimental outcomes that lie in B. Then the conditional probability of the event A given the occurrence of the event B is

Notice that (for this example) the conditional probability, P (AjB), di®ers from the unconditional probability, P (A).

To develop a de¯nition of conditional probability that is not speci¯c to ¯nite sample spaces with equally likely outcomes, we now write

We take this as a de¯nition:

De¯nition 2.1 If A and B are events, and P (B) > 0, then

Figure 2.4: Venn Diagram for Conditional Probability

The following consequence of De¯nition 2.1 is extremely useful. Upon multiplication of equation (2.5) by P (B), we obtain

P (A \ B) = P (B)P (AjB)

when P (B) > 0. Furthermore, upon interchanging the roles of A and B, we obtain

P (A \ B) = P (B \ A) = P (A)P (BjA)

when P (A) > 0. We will refer to these equations as the multiplication rule for conditional probability.

Used in conjunction with tree diagrams, the multiplication rule provides a powerful tool for analyzing situations that involve conditional probabilities.

Example 1 Consider three fair coins, identical except that one coin (HH) is Heads on both sides, one coin (HT) is heads on one side and Tails on the other, and one coin (TT) is Tails on both sides. A coin is selected at random and tossed. The face-up side of the coin is Heads. What is the probability that the face-down side of the coin is Heads?

This problem was once considered by Marilyn vos Savant in her syndicated column, Ask Marilyn. As have many of the probability problems that

she has considered, it generated a good deal of controversy. Many readers reasoned as follows:

1.The observation that the face-up side of the tossed coin is Heads means that the selected coin was not TT. Hence the selected coin was either

HH or HT.

2.If HH was selected, then the face-down side is Heads; if HT was selected, then the face-down side is Tails.

3.Hence, there is a 1 in 2, or 50 percent, chance that the face-down side is Heads.

At ¯rst glance, this reasoning seems perfectly plausible and readers who advanced it were dismayed that Marilyn insisted that :5 is not the correct probability. How did these readers err?

Figure 2.5: Tree Diagram for Example 1

A tree diagram of this experiment is depicted in Figure 2.5. The branches represent possible outcomes and the numbers associated with the branches are the respective probabilities of those outcomes. The initial triple of branches represents the initial selection of a coin|we have interpreted \at random" to mean that each coin is equally likely to be selected. The second level of branches represents the toss of the coin by identifying its resulting

up-side. For HH and TT, only one outcome is possible; for HT, there are two equally likely outcomes. Finally, the third level of branches represents the down-side of the tossed coin. In each case, this outcome is determined by the up-side.

The multiplication rule for conditional probability makes it easy to calculate the probabilities of the various paths through the tree. The probability that HT is selected and the up-side is Heads and the down-side is Tails is

P (HT \ up=H \ down=T) = P (HT \ up=H) ¢ P (down=TjHT \ up=H)

=P (HT) ¢ P (up=HjHT) ¢ 1

=(1=3) ¢ (1=2) ¢ 1

=1=6

and the probability that HH is selected and the up-side is Heads and the down-side is Heads is

P (HH \ up=H \ down=H) = P (HH \ up=H) ¢ P (down=HjHH \ up=H)

=P (HH) ¢ P (up=HjHH) ¢ 1

=(1=3) ¢ 1 ¢ 1

=1=3:

Once these probabilities have been computed, it is easy to answer the original question:

which was Marilyn's answer.

From the tree diagram, we can discern the fallacy in our ¯rst line of reasoning. Having narrowed the possible coins to HH and HT, we claimed that HH and HT were equally likely candidates to have produced the observed Head. In fact, HH was twice as likely as HT. Once this fact is noted it seems completely intuitive (HH has twice as many Heads as HT), but it is easily overlooked. This is an excellent example of how the use of tree diagrams may prevent subtle errors in reasoning.

Example 2 (Bayes Theorem) An important application of conditional probability can be illustrated by considering a population of patients at risk for contracting the HIV virus. The population can be partitioned

Figure 2.6: Tree Diagram for Example 2

To motivate our calculation of P (Dj+), it is again helpful to construct a tree diagram, as in Figure 2.6. This diagram was constructed so that the branches depicted in the tree have known probabilities, i.e. we ¯rst branch on the basis of disease state because P (D) and P (Dc) are known, then on the basis of test result because P (+jD), P (¡jD), P (+jDc), and P (¡jDc) are known. Notice that each of the four paths in the tree corresponds to exactly one of the four sets in (2.6). Furthermore, we can calculate the probability of each set by multiplying the probabilities that occur along its corresponding path:

P (D \ +) = P (D) ¢ P (+jD) = :001 ¢ :997;

P (D \¡) = P (D) ¢ P (¡jD) = :001 ¢ :003;

P (Dc \ +) = P (Dc) ¢ P (+jDc) = :999 ¢ :015;

P (Dc \¡) = P (Dc) ¢ P (¡jDc) = :999 ¢ :985:

The predictive value of the test is now obtained by computing

:001 ¢ :997

= :001 ¢ :997 + :999 ¢ :015

:

= :0624:

This probability may seem quite small, but consider that a positive test result can be obtained in two ways. If the person has the HIV virus, then a positive result is obtained with high probability, but very few people actually have the virus. If the person does not have the HIV virus, then a positive result is obtained with low probability, but so many people do not have the virus that the combined number of false positives is quite large relative to the number of true positives. This is a common phenomenon when screening for diseases.

The preceding calculations can be generalized and formalized in a formula known as Bayes Theorem; however, because such calculations will not play an important role in this book, we prefer to emphasize the use of tree diagrams to derive the appropriate calculations on a case-by-case basis.

Independence We now introduce a concept that is of fundamental importance in probability and statistics. The intuitive notion that we wish to formalize is the following:

Two events are independent if the occurrence of either is unaffected by the occurrence of the other.

This notion can be expressed mathematically using the concept of conditional probability. Let A and B denote events and assume for the moment that the probability of each is strictly positive. If A and B are to be regarded as independent, then the occurrence of A is not a®ected by the occurrence of B. This can be expressed by writing

Similarly, the occurrence of B is not a®ected by the occurrence of A. This can be expressed by writing

Substituting the de¯nition of conditional probability into (2.7) and multiplying by P (B) leads to the equation

P (A \ B) = P (A) ¢ P (B):