An Introduction To Statistical Inference And Data Analysis

Figure 7.5: Normal Probability Plots of Four Samples from Normal(0; 1)

7.4Density Estimates

Suppose that ~x = fx1; : : : ; xng is a sample drawn from an unknown pdf f. Box plots and normal probability plots are extremely useful graphical techniques for discerning in ~x certain important attributes of f, e.g. centrality, dispersion, asymmetry, nonnormality. To discern more subtle features of f, we now ask if it is possible to reconstruct from ~x a pdf f^n that approximates f. This is a di±cult problem, one that remains a vibrant topic of research and about which little is said in introductory courses. However, using the concept of the empirical distribution, one can easily motivate one of the most popular techniques for nonparametric probability density estimation.

The logic of the empirical distribution is this: by assigning probability 1=n to each xi, one accumulates more probability in regions that produced more observed values. However, because the entire amount 1=n is placed exactly on the value xi, the resulting empirical distribution is necessarily discrete. If the population from which the sample was drawn is discrete, then the empirical distribution estimates the probability mass function. But if the population from which the sample was drawn is continuous, then all

Example 7 Consider the probability density function f displayed in Figure 7.6. The most striking feature of f is that it is bimodal. Can we detect this feature using a sample drawn from f?

Figure 7.6: A Bimodal Probability Density Function

We drew a sample of size n = 100 from f. A box plot and a normal probability plot of this sample are displayed in Figure 7.7. It is di±cult to discern anything unusual from the box plot. The normal probability plot contains all of the information in the sample, but it is encoded in such a way that the feature of interest is not easily extracted. In contrast, the kernel density estimate displayed in Figure 7.8 clearly reveals that the sample was drawn from a bimodal population. After storing the sample in the vector x, this estimate was computed and plotted using the following S-Plus command:

> plot(density(x),type="b")

Figure 7.7: Box Plot and Normal Probability Plot for Example 7

7.5Exercises

1.The following independent samples were drawn from four populations:

(a)Use the boxplot function to create side-by-side box plots of these samples. Does it appear that these samples were all drawn from

the same population? Why or why not?

Figure 7.8: A Kernel Density Estimate for Example 7

(b)Use the rnorm function to draw four independent samples, each of size n = 10, from one normal distribution. Examine box plots of these samples. Is it possible that Samples 1{4 were all drawn from the same normal distribution?

2.The following sample, ~x, was collected and sorted:

0:246 0:327 0:423 0:425 0:434 0:530 0:583 0:613 0:641 1:054 1:098 1:158 1:163 1:439 1:464 2:063 2:105 2:106 4:363 7:517

(a)Graph the empirical cdf of ~x.

(b)Calculate the plug-in estimates of the mean, the variance, the median, and the interquartile range.

(c)Take the square root of the plug-in estimate of the variance and compare it to the plug-in estimate of the interquartile range. Do you think that ~x was drawn from a normal distribution? Why or

why not?

(d)Use the qqnorm function to create a normal probability plot. Do you think that ~x was drawn from a normal distribution? Why or why not?

(e)Now consider the transformed sample ~y produced by replacing each xi with its natural logarithm. If ~x is stored in the vector x, then ~y can be computed by the following S-Plus command:

>y <- log(x)

Do you think that ~y was drawn from a normal distribution? Why or why not?

3.Forty-one students taking Math 308 (Applied Statistics) at the College of William & Mary were administered a test. The following test scores were observed and sorted:

(a)Do these numbers appear to be a random sample from a normal distribution?

(b)Does this list of numbers have any interesting anomalies?

4.Experiment with using S-Plus to generate simulated random samples of various sizes. Use the summary function to compute the quartiles of these samples. Try to discern the convention that this function uses to de¯ne sample quartiles.

Chapter 8

Inference

In Chapters 2{6 we developed methods for studying the behavior of random variables. Given a speci¯c probability distribution, we can calculate the probabilities of various events. For example, knowing that Y » Binomial(n = 100; p = :5), we can calculate P (40 · Y · 60). Roughly speaking, statistics is concerned with the opposite sort of problem. For example, knowing that Y » Binomial(n = 100; p), where the value of p is unknown, and having observed Y = y (say y = 32), what can we say about p? The phrase statistical inference describes any procedure for extracting information about a probability distribution from an observed sample.

The present chapter introduces the fundamental principles of statistical inference. We will discuss three types of statistical inference|point estimation, hypothesis testing, and set estimation|in the context of drawing inferences about a single population mean. More precisely, we will consider the following situation:

1.X1; : : : ; Xn are independent and identically distributed random variables. We observe a sample, ~x = fx1; : : : ; xng.

2.EXi = ¹ and Var Xi = ¾2 < 1. We are interested in drawing inferences about the population mean ¹, a quantity that is ¯xed but unknown.

3.The sample size, n, is su±ciently large that we can use the normal approximation provided by the Central Limit Theorem.

We begin, in Section 8.1, by examining a narrative that is su±ciently nuanced to motivate each type of inferential technique. We then proceed to

147

discuss point estimation (Section 8.2), hypothesis testing (Sections 8.3 and 8.4), and set estimation (Section 8.5). Although we are concerned exclusively with large-sample inferences about a single population mean, it should be appreciated that this concern often arises in practice. More importantly, the fundamental concepts that we introduce in this context are common to virtually all problems that involve statistical inference.

8.1A Motivating Example

We consider an arti¯cial example that permits us to scrutinize the precise nature of statistical reasoning. Two siblings, a magician (Arlen) and an attorney (Robin) agree to resolve their disputed ownership of an Ert¶e painting by tossing a penny. Just as Robin is about to toss the penny in the air, Arlen suggests that spinning the penny on a table will ensure better randomization. Robin assents and spins the penny. As it spins, Arlen calls \Tails!" The penny comes to rest with Tails facing up and Arlen takes possession of the Ert¶e.

That evening, Robin wonders if she has been had. She decides to perform an experiment. She spins the same penny on the same table 100 times and observes 68 Tails. It occurs to Robin that perhaps spinning the penny was not entirely fair, but she is reluctant to accuse her brother of impropriety until she is convinced that the results of her experiment cannot be dismissed as coincidence. How should she proceed?

It is easy to devise a mathematical model of Robin's experiment: each spin of the penny is a Bernoulli trial and the experiment is a sequence of n = 100 trials. Let Xi denote the outcome of spin i, where Xi = 1 if Heads is observed and Xi = 0 if Tails is observed. Then X1; : : : ; X100 » Bernoulli(p), where p is the ¯xed but unknown (to Robin!) probability that a single spin will result in Heads. The probability distribution Bernoulli(p) is our mathematical abstraction of a population and the population parameter of interest is ¹ = EXi = p, the population mean.

Let

X100

Y = Xi;

i=1

the total number of Heads obtained in n = 100 spins. Under the mathematical model that we have proposed, Y » Binomial(p). In performing her