CFA Level 1 (2009) - 1

Study Session 3

Cross-Reference to CFA Institute Assigned Reading #10 - Sampling and Estimation

LOS 10.f: Distinguish between a point estimate and a confidence interval estimate of a population parameter.

LOS 10.h: Explain the construction of confidence intervals.

Point estimates are single (sample) values used to esrimare popular ion paramerers. The formula used to compure rhe poinr esrimare is called rhe esrimator. For example, rhe sample mean, x, is an esrimaror of rhe popularion mean 1-1. and is compured using rhe

familiar formula:

The value generared wirh rhis calcularion for a given sample is called rhe point estimate of rhe mean.

Confidence inrerval esrimares resulr in a range of values wirhin which rhe acmal value or a paramercr \"illlie, given rhe prohabiliry of I - 1\. Hat:, alpha. (I. is called rhc 10'1'1 I~r.iig"ific{/ncc for rhe confidence inrerval. and rhe probabiliry I - (\ is referred [() as

rhe degree ofconfidence. For example, we mighr esrimare rhar rhe popularion mean of random variables will range from 15 to 25 wirh a 95% degree of confidence, or at rhe "i% level of significance.

Confidence inrervals are usually consr:rucred by adding or subrracring an appropriate value from rhe poinr esrimare. In general, confidence inrervals rake on rhe following form:

point esrimare ± (reliability factor x srandard error)

LOS 10.g: ldentify and describe the desirable properties of an estimator.

Regardless of wherher we arc concerned wirh poinr esrimares or confidence intervals, rhere arc certain sratisrical properries rhat make some esrimates more desirable than others. These desirable properties of an estimator are unbiasedness, efficiency, and consistency.

• An unbiased esrimaror is one for which rhe expecred value of rhe esrimator is equal ro rhe parameter you are trying to esrimare. For example, because rhe expecred value of rhe sample mean is equal to the popularion mean [E(x) = 1-1.], rhe sample mean is an unbiased esrimaror of the popularion mean.

Srudy Session 3

Cross-Reference to CFA Institute Assigned Reading #10 - Sampling and Estimation

•An unbiased estimator is also efficient if the variance of its sampling distribution is smaller than all the other unbiased estimators of the parameter you are trying to estimate. The sample mean, for example, is an unbiased and efficient estimator of the population mean.

A consistent estimator is one for which the accuracy of the parameter estimate increases as the sample size increases. As the sample size increases, the standard error of the sample mean falls, and the sampling distribution bunches more closely around the population mean. In fact, as the sample size approaches infinity, the standard error approaches zero.

LOS 10.i: Describe the properties of Student's t-distribution, and calculate and interpret its degrees of freedom.

Student's t-distribution. or simply the t-distribution, is a bell-shaped probability distribution that is symmetrical about its mean. It is the appropriate distribution to use when constructing confidence inrervals based on mlflll i"amples (n < ;:\0) from populations with IIl/knOIl!11 l'al"itllicc and a normal. or approxin13tely normal, distribution. It may

also be al)propriatc' to use the {-distributiOIl whell rhe population variance is unknown and the sample size is large enough that the central limit theorem will assure dut the sampling distribution is approximately normal.

Srudenr's t-distribution has the ti)llowing properties:

•It is symmetrical.

•It is defined by a single parameter, the degrees of freedom (df), where the degrees of freedom are equal to the number of sample observations minus 1, n - J, for sample means,

•It is less peaked than a normal distribution. with more probability in the tails ("fatter tails").

•As the degrees of freedom (the sample size) gets larger, the shape of the t-distribution more closely approaches a standard normal distribution.

When compared to the normal distribution, the t-distribution is flatter with more area under the tails (i.e., it has fatter tails). As the degrees of freedom for the t-distribution increase. however. its shape approaches that of the normal distribution.

The degrees of freedom for tests based on sample means are n - 1 because, given the mean. only n - 1 observations can be unique.

The t-distribution is a symmetrical distribution that is centered about zero. The shape of the t-distribution is dependent on the number of degrees of freedom, and degrees of freedom are based 011 the number of sJmple observations. The t-distribution is flarrer and has thicker tails than the standard normal distribution. As the number of

observations increases (i.e., the degrees of freedom increase), the t-distribution becomes more spiked and its tails become thinner. As the number of degrees of freedom increases without bound, the t-distriburion converges to the standard normal distribution

(z-distribu tion). The thickness of the tails relative to those of the z-distribution is important in hypothesis testing because thicker tails mean more observations away from the center of the distribution (i.e., more "outliers"). Hence, hypothesis testing using

the t-distribution makes it more difficult to reject the null relative to hypothesis testing using the z-distribution.

Study Session 3

Cross-Reference to CFA Institute Assigned Reading #10 - Sampling and Estimation

The table in Figure 1 contains one-tailed critical values for the t-distribution at the 0.05 and 0.025 levels of significance with various degrees of freedom (df). Note that, unlike the z-table, the t-values are contained within the table, and the probabilities are located at the column headings. Also note that the level of significance of a t-test corresponds to the one-tailed probabilities, p, that head the columns in the t-table.

Figure 2 illustrates the different shapes of the t-distribution associated with different degrees of freedom. The tendency is for the t-distribution to become more peaked and to look more and more like the normal distribution as the degrees of freedom increase. Practically speaking, the greater the degrees of freedom, the greater the percentage

of observations near the center of the distribution and the lower the percentage of observations in the tails, which are thinner as degrees of freedom increase. This means that confidence intervals for a random variable that follows a t-distribution must be wider (narrower) when the degrees of freedom are less (more) for a given significance level.

Figure 1: Table of Critical t- Values

Page 274

Study Session 3

Cross-Reference to CFA Institute Assigned Reading #10 - Sampling and Estimation

Figure 2: t-Distributions for Different Degrees of Freedom (df)

o

LOS IO.j: Calculate and interpret a confidence interval for a population mean, giver, ;'. normal distribution with]) a known population variance, 2) an

unknown population variance, or 3) witb an unknown variance and the sample size i~ large.

If the population has a normal distrilmtioll with fl known variance, a confidence interval for the population mean can be calculated as:

reLiabilit)I factor, a standard normal random variable for which the probability in the right-hand tail of the distribution is 0./2. In other

words, this is the z-score that leaves 0./2 of probability in the upper tail.

CT

the standard error of the sample mean where CT is the known standard deviation of the population, and 11 is the sample size.

The most commonly used standard normal distribution reliability factors are:

I(\/2 = 1.645 for 90% confidence intervals (the significance level is 10%, 5% in each tail).

zn/:' 1.960 for 9YY<J confidence intervals ([he significance level is 5%, 2.5% in each tail).

zn/2 2.575 for 99% confidence intervals (the significance level is 1%,0.5% in each tail).

Do these numbers look familiar? They should! In our review of common probability distributions, we found the probability under the standard normal curve between

z = -1.96 and z = +1.96 to be 0.95, or 95%. Owing to symmetry, this leaves a

Study Session 3

Cross-Reference to CPA Institute Assigned Reading #10 - Sampling and Estimation

'C~~j~~i;FtJ.fidenceinterva1 . " .. :':5-,,:.,. . .'

.c~iiSid:df-~lpractice exam that was administered to3~:L#vel,1 ~andidates. The mean ::,S~Q~;,o.~3~W:practiceexam was 80. Assuming-apoP41~tIOn:standaiddeviation equal

::i0';~ii~~b'ttStl:uttand interpret a 99%confi.dence in.teryaffor the. mean score on the •.~p·~~~~t~~riiJor~6:c.andidaies~ :Notethatin_this;~xaiijle.t~e population·standard

~- ,'

Ai:a,corifidencelevel of 99%, Zo./2 = zO.005 = 2.58. So, the 99% confidence interval is calctila:1:ed as follows: .

x±zon ~ = 80±2.58 ~ = 80±6.45

Thus, the 99% confidence interval ranges from 73.55 to 86.45.

Confidence intervals can be interpreted from a probabilistic perspective or a practical perspective. \X'ith regard to the outcome of the CFA® practice exam example, dJcse two perspectives C:l.n b-;:: described as follows:

Probabilistic interpretation. After repeatedly taking samples of CFA candidates, administering the practice exam, and constructing confidence intervals for each sample's mean, 99% of the resulting confidence intervals will, in the long run, include the population mean.

•Practical interpretation. We are 99% confident that the population mean score is between 73.55 and 86.45 for candidates from this population.

Confidence Intervals for the Population Mean: Normal With Unknown Variance

If the distribution of the population is normal with unknown variance, we can use the t-distribution to construct a confidence interval:

where:

the point estimate of the population mean

the t-reliability facror (a.k.a. t-statistic or critical t-value) corresponding (() a t-distributed random variable with n - 1 degrees of freedom, where n is the sample size. The area under the tail of the t-distribution to the right of t

is 0./2

s

~standard error of the sample mean.

s sample standard deviation.

Study Session 3

Cross-Reference to CFA Institute Assigned Reading #10 - Sampling and Estimation

Unlike the standard normal distribution, the reliability factors for the t-distribution depend on the sample size, so we can't rely on a commonly used set of reliability factors. Instead, reliability factors for the t-disuibution have to be looked up in a table of Student's t-disuibution, like the one at the back of this book.

Owing to the relatively fatter tails of the t-distribution, confidence intervals constructed

llsing t-reliability factors (t I~) will be more conservative (wider) than those constructed

f\ _

using z-reliabili ty factors (Zo./2)'

Example: Confidence intervals

Let'sreturl1'to the McCreary, Inc. example. Recall thatw~ took a sample~frhepast

30 mornhly stock returns for McCreary, Inc. and determined that the nieaineturn . was 2% and the sample standard deviation was 20%~Since.thepopulationvarian~eis

Now, let's construct a 95% confidence interval for the mean monthly rerum.

Answer:

Here we will use the t-reliability factor because the population variance is unknown. Since there are 30 observations, the degrees of freedom are 29 = 30 - 1. Remember, because this is a two-tailed test at the 95% confidence level, the probability under each tail must be 0../2 = 2.5%, for a total of 5%. So, referencing the one-tailed probabilities for Student's t-distribution at the back of this book, we find the critical t-value (reliability factor) for a/2 :: 0.025 and df = 29 to be t29, 2.5 = 2.045. Thus, the 95% confidence interval for the population mean is:

2% ± 2.045( ~J= 2% ± 2.045(3.6%) = 2% ± 7.4%

Thus, the 95% confidence has a lower limit of -5.4% and an upper limit of +9.4%.

We can interpret this confidence interval by saying that we are 95% confidennhat the population mean monthly return for McCreary stock is between -5.4% and +9.4%.

Professors Note: .You should practice looking up reliability factors (a.k.a. critical t-values or t-statistics) in a t-table. The first step is always to compute the degrees offreedom, which is n -1. The second step-is to find the appropriate level ofalpha or significance. This depends on whether the test you're concerned with is one-tailed (use a) or two'-tailed (use 0.12). In this review, our tests will always be two-tailed because confidence intervals are designed to compute an upper and lowerlimit. Thus, We will use 0.12. To look upt29, 2.5, find the 29 dfrowandmatehit with the ..

0.025 column; t =2:045 is the result. We'ildo more ofthis in our studyof hypothesis testing. . . ..

Study Session 3

Cross-Reference to CFA Institute Assigned Reading #10 - Sampling and Estimation

Confidence Interval for a Population Mean When the Population Variance Is

Unknown Given a Large Sample From Any Type of Distribution

We now know that the z-statistic should be used to construct confidence intervals when the population distribution is normal and the variance is known, and the t-statistic should be used when the distribution is normal but the variance is unknown. But what do we do when the distribution is nonnormal?

As it turns out, the size of the sample influences whether or not we can construct the appropriate confidence interval for the sample mean.

If the distribution is non normal but the population variance is known, the z-statistic can be used as long as the sample size is large (n ~ 30). We can do this because the central limit theorem assures us that the distribution of the sample mean is approximately normal when the sample is large.

•If the distribution is nOll/lOrma! and the population variance is Illlkllou1n, the t-statistic

can be used as long as the sample size is large (n 2 30). It is J,Jso acceptable to use the :-sratistic although use of the t-slJ,tistic is more conservative.

This means that if we arc sampling from a nonnormaJ distribution (which is sometimes the case in finance), we cannot create a confidence interval ift!;e sample size is less than 30.

So, all else equal, make sure you have a sample of at least 30, and the larger, the better.

Figure 3 summarizes this discussion.

o Pro.fess01·J Note: rou should commit the criteria in this table to mem01Y

Figure 3: Criteria for Selecting the Appropriate Test Statistic

* The z-statistic is theoretically acceptable here, but use of the t-statistic is more conservative.

All of the preceding analysis depends on the sample we draw from the population being random. If the sample isn't random, the central limit theorem doesn't apply, our estimates won't have the desirable properties, and we can't form unbiased confidence intervals. Surprisingly, creating a random sample is not as easy as one might believe. There are a number of potential mistakes in sampling methods that can bias the results.

Study Session 3

Cross-Reference to CFA lnstitu~e Assigned Reading #10 - Sampling and Estimation

These biases are particularly problematic in financial research, where available historical data are plentiful, bur the creation of new sample data by experimentation is restricted.

LOS 10.k: Discuss the issues regarding selection of the appropriate sample size, data-mining bias, sample selection bias, survivorship bias, look-ahead bias, and time-period bias.

We have seen so far that a larger sample reduces the sampling error and the standard deviation of the sample statistic around its true (population) value. Confidence intervals are narrower when samples are larger and the standard errors of the point estimates of population parameters are less.

There are two limitations on this idea of "larger is bener" when it comes to selecting an appropriate sample size. One is that larger samples may contain observations from a different population (distribution). If we include observations which come from a

diffclclH population (one with a differcnr population paramctcr). we will not necessarily improve. and may even reduce. the precision of our population parameter estinutrs. The other consider,nion is cost. The costs of using a larger sample must be weighed against the value of the increase in precision from the incrcase in sample size. Both of these factors suggest that the largest possible sample size is not always the most appropriate choice.

Data-Mining Bias, Sample Selection Bias, Survivorship Bias, Look-Ahead Bias, and Time-Period Bias

Data mining occurs when analysts repeatedly use the same database to search for patterns or trading rules until one that "works" is discovered. For example, empirical research has provided evidence that value stocks appear to outperform growth stocks. Some researchers argue that this anomaly is acrually the product of data mining. Because the data set of historical stock rerurns is quite limited, it is difficult to know for sure whether the difference between value and growth srock rerurns is a true economic phenomenon, or simply a chance pattern that was stumbled upon after repeatedly looking for any identifiable panern in the d;lta.

Data-mining bias refers to results where the statistical significance of the pattern is overestimated because the results were found through data mining.

When reading research findings that suggest a profitable trading strategy, make sure you heed the following warning signs of data mining:

Evidence that many different variables were tested, most of which are unreported, until significant ones were found.

•The lack of any economic theory that is consistent with the empirical results.

The best way to avoid data mining is to test a potentially profitable trading rule on a data set different from the one you used to develop the rule (i.e., use out-oE-sample data) .

Study Session 3

Cross-Reference to CFA Institute Assigned Reading #10 - Sampling and Estimation

Sample selection bias occurs when some data is systematically excluded from the analysis, usually because of the lack of availabiliry. This practice renders the observed sample to be nonrandom, and any conclusions drawn from this sample can't be applied to the population because the observed sample and the portion of the population that was not observed are different.

Survivorship bias is the most common form of sample selection bias. A good example of the existence of survivorship bias in investments is the study of mutual fund performance. Most mutual fund databases, like Morningstar®'s, only include funds currently in existence-the "survivors." They do not include funds that have ceased to exist due to closure or merger.

This would not be a problem if the characteristics of the surviving funds and the missing funds were the same; then the sample of survivor funds would still be a random sample drawn from the population of mutual funds. As one would expect, however, and as evidence has shown, the funds that are dropped from the sample have lower returns relative ro rhe surviving funds. Thus, the surviving sample is biased toward the hetter runds (i.e., it is not random). The analysis ora lIlutual fund sample with survivorship bias will yield results that overesrimart' rhe average mutual funo rerurn because rhe darabase only includes the better-performing funds. The solution to survivorship bias is ro use a sample of funds that all started ;u the same time and not drop funds rhar have been droppeo from rhe sample.

Look-ahead bias occurs when a stlldy tests a relationship using sample dara thar was nor available on rhe rest date. For example, consider rhe rest of a trading rule rhat is based on rhe price-to-book ratio at the end of the fiscal year. Srock prices are available for all companies at the same point in rime, while end-of-year book values may nor be available until 30 to GO days afrer the fiscal year ends. In order to accounr for this bias, a study that uses price-to-book value ratios to tesr trading strategies might estimate the book value as reported at fiscal year end and the market value two months later.

Time-period bias can result if the time period over which the dara is gathered is

either roo short or too long. If rhe time period is too shon, research resulrs may reflect phenomena specific to thar rime period, or perhaps even data mining. If the time period is too long, rhe fundamental economic relationships rhar underlie the results may have changed.

For example, research findings may indicate that small srocks ourperformed large srocks during 1980-1985. This may well be rhe result of rime-period bias-in this case, using roo short a time period. It's not clear whether this relarionship will continue in the future or if it is just an isolated occurrence.

On the other hand, a study that quantifies the relationship between inflation and unemployment (rhe Phillips Curve) during rhe period from 1940-2000 will also result in time-period bias-because this period is too long, and it covers a fundamental change in the relationship between inflation and unemployment that occurred in the 1980s. In this case, the data should be divided inro two subsamples rhat span rhe period before and after the change.