vstatmp_engl

-Very useful especially for the solution of numerical problems is a book by Blobel and Lohrman “Statistische und numerische Methoden der Datenanalyse” [8] written in German.

-Other useful books written by particle physicists are found in Refs. [9, 10, 11, 12]. The books by Roe and Barlow are more conventional while Cowan and D’Agostini favor a moderate Bayesian view.

-Modern techniques of statistical data analysis are presented in a book written by professional statisticians for non-professional users, Hastie et al. “The Elements of Statistical Learning”[13]

-A modern professional treatment of Bayesian statistics is the textbook by Box and Tiao “Bayesian Inference in Statistical Analysis” [14].

The interested reader will find work on the foundations of statistics, on basic principles and on the standard theory in the following books:

-Fisher’s book [15] “Statistical Method, Experimental Design and Scientific Inference” provides an interesting overview of his complete work.

-Edward’s book “Likelihood” [16] stresses the importance of the likelihood function, contains many useful references and the history of the Likelihood Principle.

-Many basic considerations and a collection of personal contributions from a moderate Bayesian view are contained in the book “Good Thinking” by Good [17]. A collection of work by Savage [18], presents a more extreme Baysian point of view.

-Somewhat old fashioned textbooks of Bayesian statistic which are of historical interest are the books of Je reys [19] and Savage [20].

Recent statistical work by particle physicists and astrophysicists can be found in the proceedings of the PHYSTAT Conferences [21] held during the past few years. Many interesting and well written articles can be found also in the internet.

This personal selection of literature is obviously in no way exhaustive.

2

Basic Probability Relations

2.1 Random Events and Variables

Events are processes or facts that are characterized by a specific property, like obtaining a “3” with a dice. A goal in a soccer play or the existence of fish in a lake are events. Events can also be complex facts like rolling two times a dice with the results three and five or the occurrence of a certain mean value in a series of measurements or the estimate of the parameter of a theory. There are elementary events, which mutually exclude each other but also events that correspond to a class of elementary events, like the result greater than three when throwing a dice. We are concerned with random events which emerge from a stochastic process as already introduced above.

When we consider several events, then there are events which exclude each other and events which are compatible. We stick to our standard example dice. The elementary events three and five exclude each other, the events greater than two and five are of course compatible. An other common example: We select an object from a bag containing blue and red cubes and spheres. Here the events sphere and cube exclude each other, the events sphere and red may be compatible.

The event A is called the complement of event A if either event A or event A applies, but not both at the same time (exclusive or ). Complementary to the event three in the dice example is the event less than three or larger than three (inclusive or ). Complementary to the event red sphere is the event cube or blue sphere.

The event consisting of the fact that an arbitrary event out of all possible events applies, is called the certain event. We denote it with Ω. The complementary event is the impossible event, that none of all considered events applies: It is denoted with, thus = Ω.

Some further definitions are useful: Definition 1: A B means A or B.

The event A B has the attributes of event A or event B or those of both A and B (inclusive or ). (The attribute cube red corresponds to the complementary event

blue sphere.)

Definition 2: A ∩ B means A and B.

10 2 Basic Probability Relations

The event A ∩ B has the attributes of event A as well as those of event B. (The attribute cube ∩ red corresponds to red cube.) If A ∩ B = , then A, B mutually exclude each other.

Definition 3: A B means that A implies B.

It is equivalent to both A B = B and A ∩ B = A. From these definitions follow the trivial relations

For any A, B we have

A B = A ∩ B , A ∩ B = A B .

To the random event A we associate the probability P {A} as discussed above. In all practical cases random events can be identified with a variable, the random variable or variate. Examples for variates are the decay time in particle decay, the number of cosmic muons penetrating a body in a fixed time interval and measurement errors. When the random events involve values that cannot be ordered, like shapes or colors, then they can be associated with classes or categorical variates.

2.2 Probability Axioms and Theorems

•Axiom 3 P {A B} = P {A} + P {B} if A ∩ B =

The probability that A or B applies is equal to the sum of the probabilities that A or that B applies, if the events A and B are mutually exclusive.

These axioms and definitions imply the following theorems whose validity is rather obvious. They can be illustrated with so-called Venn diagrams, Fig. 2.1. There the areas of the ellipses and their intersection are proportional to the probabilities.

P {A} = 1 − P {A} , P { } = 0 ,

P {A B} = P {A} + P {B} − P {A ∩ B} ,

1They are called Kolmogorov axioms, after the Russian mathematician A. N. Kolmogorov (1903-1987).

Relation (2.1) together with (2.2) and axioms 1, 2 imply 0 ≤ P {A} ≤ 1. For arbitrary events we have

P {A B} ≥ P {A} , P {B} ; P {A ∩ B} ≤ P {A} , P {B} .

If all events with the attribute A possess also the attribute B, A B, then we have

P {A ∩ B} = P {A}, and P {A B} = P {B}.

2.2.2 Conditional Probability, Independence, and Bayes’ Theorem

In the following we need two further definitions:

Definition: P {A | B} is the conditional probability of event A under the condition that B applies. It is given, as is obvious from Fig. 2.1, by:

A conditional probability is, for example, the probability to find a sphere among the red objects. The notation A | B expresses that B is considered as fixed, while A is the random event to which the probability refers. Contrary to P {A}, which refers to arbitrary events A, we require that also B is valid and therefore P {A ∩ B} is normalized to P {B}.

Among the events A | B the event A = B is the certain event, thus P {B | B} = 1. More generally, from definition 3 of the last section and (2.3) follows P {A|B} = 1 if B implies A:

B A A ∩ B = B P {A|B} = 1 .

Definition: If P {A ∩ B} = P {A} × P {B}, the events A and B (more precisely: the probabilities for their occurrence) are independent.

From (2.3) then follows P {A | B} = P {A}, i.e. the conditioning on B is irrelevant for the probability of A. Likewise P {B | A} = P {B}.

In Relation (2.3) we can exchange A and B and thus P {A | B}P {B} = P {A ∩

Bayes’ theorem is frequently used to relate the conditional probabilities P {A | B} and P {B | A}, and, as we will see, is of some relevance in parameter inference.

The following simple example illustrates some of our definitions. It assumes that each of the considered events is composed of a certain number of elementary events which mutually exclude each other and which because of symmetry arguments all have the same probability.

Example 2. Card game, independent events

The following table summarizes some probabilities for randomly selected cards from a card set consisting of 32 cards and 4 colors.

From the proportion of students and women in the population and the fraction of students among women we compute the fraction of women among students:

The dependence of the events A and B manifests itself in the di erence of P {A} and P {A | B}. Applying Bayes’ theorem we obtain

P {B | A} = P {A | B}P {B} P {A}

= 0.018 · 0.5 = 0.45 . 0.02

About 45% of the students are women.

Example 5. Bayes’ theorem, beauty filter

The probability P {A} that beauty quark production occurs in a colliding beam reaction be 0.0001. A filter program selects beauty reactions A with e ciency P {b | A} = 0.98 and the probability that it falsely assumes that beauty is present if it is not, be P {b | A} = 0.01. What is the probability P {A | b} to have genuine beauty production in a selected event? To solve the problem, first the probability P {b} that a random event is selected has to be evaluated,

where the bracket in the first line is equal to 1. In the second line Bayes’ theorem is applied. Applying it once more, we get

P {A | b} = P {b | A}P {A}

P {b}

P {b | A}P {A}

=

P {b | A}P {A} + P {b | A}P {A}

0.98 · 0.0001 = 0.98 · 0.0001 + 0.01 · 0.9999 = 0.0097 .

About 1% of the selected events corresponds to b quark production.

Bayes’ theorem is rather trivial, thus the results of the last two examples could have easily been written down without referring to it.