vstatmp_engl

From these so-called normal equations we get the estimate for the P parameters ˆ :

θ

ˆ = ( T )−1 T ′ . (7.12)

θ T VT T Vy

Note that TT VT is a symmetric P × P matrix with unique inverse which turns out to be the error (i.e. covariance) matrix of θ: With the usual propagation of errors we obtain (see Sect. 4.3):

Eθ = DCDT ,

= (TT VT)−1 ,

substituting for D the matrix on the right hand side of (7.12), after some simplifications.

Linear regression provides an optimal solution only for normally distributed errors. If the distribution of errors is not known up to their variance4, it is according to the Gauss–Markov theorem optimal in the following restricted sense: If the error distribution has mean value zero, the estimator is unbiased and among all other unbiased estimators which are linear in the observations its estimator has minimal variance. In fact, the result then coincides with that of a likelihood fit if normally distributed errors are assumed.

Linear problems are rare. When the prediction is a non-linear function of the parameters, the problem can be linearized by a Taylor expansion as a first rough approximation. By iteration the precision can be improved.

The importance of non-linear parameter inference by iterative linear regression has decreased considerably. The minimum searching routines which we find in all computer libraries are more e cient and easier to apply. Some basic minimum searching approaches are presented in Appendix 13.9.

7.3 Comparison of Estimation Methods

The following table contains an evaluation of the virtues and properties of the estimation approaches which we have been discussing.

Whenever possible, the likelihood method should be applied. It requires a sample of observations and a p.d.f. in analytic or well defined numerical form and is very sensitive to wrongly assigned observations in the sample. When the theoretical description of the data is given in form of a simulated histogram, the Poisson likelihood adjustment of the simulation to the bin content should be chosen. When we have to fit a function to measured data points, we use the least square method. If computing time is a limitation like in some on-line applications, the moments method lends itself. In many situations all three methods are equivalent.

All methods are sensitive to spurious background. Especially robust methods have been invented to solve this problem. An introduction and references are given in Appendix 13.15. For completeness we present in Appendix 13.3.1 some frequentist criteria of point and interval estimation which are relevant when parameters of many objects of the same type, for instance particle tracks, are measured. In the Appendix 13.6 we discuss the virtues of di erent point and interval inference approaches. Algorithms for minimum search are sketched in Appendix 13.9.

4Usually, when the variance is known, we have also an idea about the the shape of the distribution.

8

Interval Estimation

8.1 Introduction

In Chap. 4 we had presented a short introduction into error calculus. It was based on probability theory. In principle, error estimation is an essential part of statistics and of similar importance as parameter estimation. Measurements result from point estimation of one or several parameters, measurement errors from interval1 estimation. These two parts form an ensemble and have to be defined in an consistent way.

As we have already mentioned, the notation measurement error used by scientists is somewhat misleading, more precise is the term measurement uncertainty. In the field of statistics the common term is confidence intervals, an expression which often is restricted to the specific frequentist intervals as introduced by Neyman which we sketch in the Appendix.

It is in no way obvious how we ought define error or confidence intervals and this is why statisticians have very di erent opinions on this subject. There are various conventions in di erent fields of physics, and particle physicists have not yet adopted a common solution.

Let us start with a wish list which summarizes the properties in the single parameter case which we would like to realize. The extrapolation to several parameters is straight forward.

1.Error intervals should contain the wanted true parameter with a fixed probability.

2.For a given probability, the interval should be as short as possible.

3.The error interval should represent the mean square spread of measurements around the true parameter value. In allusion to the corresponding probability term we talk about standard deviation errors.

4.The definition has to be consistent, i.e. observations containing identical information about the parameters should lead to identical intervals. More precise measurements should have shorter intervals than less precise ones. The error interval has to contain the point estimate.

5.Error intervals should be invariant under transformation of the estimated parameter.

1The term interval is not restricted to a single dimension. In n dimensions it describes a n-dimensional volume.

202 8 Interval Estimation

6.The computation of the intervals should be free from subjective, e.g. more or less arbitrary model depending assumptions.

7.A consistent method for the combination of measurements and for error propagation has to exist.

8.The approach has to be simple and transparent.

Unfortunately it is absolutely impossible to fulfil simultaneously all these conditions which partially contradict each other. We will have to set priorities and sometimes we will have to use ad hoc solutions which are justified only from experience and common sense. Under all circumstances, we will satisfy point 4, i.e. consistency. As far as possible, we will follow the likelihood principle and derive the interval limits solely from the likelihood function.

It turns out that not always the same procedure is optimum for the interval estimation. For instance, if we measure the size or the weight of an object, precision is the dominant requirement, i.e. properties denoting the reliability or reproducibility of the data. Here, a quantity like the variance corresponding to the mean quadratic deviation is appropriate to describe the error or uncertainty intervals. Contrary, limits, for instance of the mass of a hypothetical particle like the Higgs particle, will serve to verify theoretical predictions. Here the dominant aspect is probability and we talk about confidence or credibility intervals2. Confidence intervals are usually defined such that they contain a parameter with high probability, e.g. 90% or 95% while error intervals comprise one standard deviation or something equivalent. The exact calculation of the standard deviation as well as that of the probability that a parameter is contained inside an interval require the knowledge of its p.d.f. which depends not only on the likelihood function but in addition on the prior density which in most cases is unknown. To introduce a subjective prior, however, is something which we want to avoid.

First we treat situations where the aspect precision dominates. There, as far as possible, we will base our considerations on the likelihood function only. Then we will discuss cases where the probability aspect is important. These will deal mainly with limits on hypothetical quantities, like masses of SUSY particles. There we will be obliged to include prior densities.

8.2 Error Intervals

The purpose of error intervals is to document the precision of a measurement. They are indispensable when we combine measurements. The combination of measurements permits us to improve continuously the precision of a parameter estimate with increasing number of measurements.

If the prior density is known with su cient precision, we determine the probability density of the parameter(s) and subsequently the moments. But this condition is so rarely fulfilled that we need not discuss it. Normally, we are left with the likelihood function only.

In what follows we will always assume that the likelihood function is of simple shape, di erentiable, with only one maximum and decreasing continuously to all

2The term credibility interval is used for Bayesian intervals.

sides. This condition is realized in most cases. In the remaining ones where it is of complicated shape we have to renounce the simple parametrization by point and interval estimates and present the full likelihood function.

The width of the likelihood function indicates how precise a measurement is. The standard error limits, as introduced in Sect. 6.5.1 – decrease by a factor e1/2 from the maximum – rely on the likelihood ratio. These limits have the positive property of being independent of the parameter metric: This means in the one-dimensional case that for a parameter λ(θ) which is a monotonic function of θ that the limits λ1, λ2, θ1, θ2 fulfill the relations λ1 = λ(θ1) and λ2 = λ(θ2). It does not matter whether we write the likelihood as a function of θ or of λ.

In large experiments usually there are many di erent e ects which influence the final result and consequently also many di erent independent sources of uncertainty, most of which are of the systematic type. Systematic errors (see Sect. 4.2.3) such as calibration uncertainties can only be treated in the Bayesian formalism. We have to estimate their p.d.f. or at least a mean value and a standard deviation.

8.2.1 Parabolic Approximation

The error assignment is problematic only for small samples. As is shown in Appendix 13.3, the likelihood function approaches a Gaussian with increasing size of the sample. At the same time its width decreases with increasing sample size, and we can neglect possible variations of the prior density in the region where the likelihood is significant. Under this condition we obtain a normally distributed p.d.f. for the parameter(s) with a standard deviation error interval given by the e1/2 decrease from the maximum. It includes the parameter with probability 68.3 % (see Sect. 4.5) The log-likelihood then is parabolic and the error interval corresponds to the region within which it decreases from its maximum by a value of 1/2 as we had fixed it already previously. This situation is certainly realized for the large majority of all measurements which are published in the Particle Data Book [24].

In the parabolic approximation the MLE and the expectation value coincide, as well as the likelihood ratio error squared and the variance. Thus we can also derive the standard deviation δθ from the curvature of the likelihood function at its maximum. For a single parameter we can approximate the likelihood function by the expression

Consequently, a change of ln Lp by 1/2 corresponds to the second derivative of L at

ˆ:

θ

We obtain the symmetric weight matrix3 V from the derivatives

3It is also called Fisher-information.