Econometrics2011

If I1 = 1 and I2 < 1 then we de…ne Eg(X) = 1: If I1 < 1 and I2 = 1 then we de…ne Eg(X) = 1: If both I1 = 1 and I2 = 1 then Eg(X) is unde…ned.

Since E(a + bX) = a + bEX; we say that expectation is a linear operator.

For m > 0; we de…ne the m0th moment of X as EXm and the m0th central moment as

E(X EX)m :

Two special moments are the mean = EX and variance 2 = E(X )2 = EX2 2: We p

call = 2 the standard deviation of X: We can also write 2 = var(X). For example, this allows the convenient expression var(a + bX) = b2 var(X):

The moment generating function (MGF) of X is

M( ) = Eexp ( X) :

The MGF does not necessarily exist. However, when it does and EjXjm < 1 then

More generally, the characteristic function (CF) of X is

C( ) = Eexp (i X)

where i = p 1 is the imaginary unit. The CF always exists, and when EjXjm < 1

kXkp = (EjXjp)1=p :

B.4 Gamma Function

The gamma function is de…ned for > 0 as

Z 1

( ) = x 1 exp ( x) :

0

It satis…es the property

(1 + ) = ( )

so for positive integers n;

(n) = (n 1)!

Special values include

(1) = 1

Sterling’s formula is an expansion for the its logarithm

Z

fX (x) = dxFX (x) = 1 f(x; y)dy:

Similarly, the marginal density of Y is

Z 1

fY (y) = f(x; y)dx:

1

The random variables X and Y are de…ned to be independent if f(x; y) = fX (x)fY (y): Furthermore, X and Y are independent if and only if there exist functions g(x) and h(y) such that f(x; y) = g(x)h(y):

if the expectations exist. For example, if X and Y are independent then

E(XY ) = EXEY:

Another implication of (B.5) is that if X and Y are independent and Z = X + Y; then

MZ( ) = Eexp ( (X + Y ))

=E(exp ( X) exp ( Y ))

=Eexp 0X Eexp 0Y

The covariance between X and Y is

cov(X; Y ) = XY = E((X EX) (Y EY )) = EXY EXEY:

The correlation between X and Y is

XY

corr (X; Y ) = XY = x Y :

The correlation is a measure of linear dependence, free of units of measurement.

If X and Y are independent, then XY = 0 and XY = 0: The reverse, however, is not true. For example, if EX = 0 and EX3 = 0, then cov(X; X2) = 0:

A useful fact is that

var (X + Y ) = var(X) + var(Y ) + 2 cov(X; Y ):

An implication is that if X and Y are independent, then

var (X + Y ) = var(X) + var(Y );

the variance of the sum is the sum of the variances.

A k 1 random vector X = (X1; :::; Xk)0 is a function from S to Rk: Let x = (x1; :::; xk)0 denote a vector in Rk: (In this Appendix, we use bold to denote vectors. Bold capitals X are random vectors and bold lower case x are nonrandom vectors. Again, this is in distinction to the notation used in the bulk of the text) The vector X has the distribution and density functions

For a measurable function g : Rk ! Rs; we de…ne the expectation

Z

Eg(X) = g(x)f(x)dx

Rk

where the symbol dx denotes dx1 dxk: In particular, we have the k 1 multivariate mean

= EX

and k k covariance matrix

0

= E (X ) (X )

= EXX0 0

If the elements of X are mutually independent, then is a diagonal matrix and

Xk ! Xk

B.7 Conditional Distributions and Expectation

The conditional density of Y given X = x is de…ned as

f(x; y) fY jX (y j x) = fX(x)

if fX(x) > 0: One way to derive this expression from the de…nition of conditional probability is

@2 F (x; y)

=@x@y

fX(x)

=f(x; y): fX(x)

The conditional mean or conditional expectation is the function

Z 1

m(x) = E(Y j X = x) = yfY jX (y j x) dy:

1

The conditional mean m(x) is a function, meaning that when X equals x; then the expected value of Y is m(x):

Similarly, we de…ne the conditional variance of Y given X = x as

=E (Y m(x))2 j X = x

=E Y 2 j X = x m(x)2:

Evaluated at x = X; the conditional mean m(X) and conditional variance 2(X) are random variables, functions of X: We write this as E(Y j X) = m(X) and var (Y j X) = 2(X): For example, if E(Y j X = x) = + 0x; then E(Y j X) = + 0X; a transformation of X:

The following are important facts about conditional expectations.

Simple Law of Iterated Expectations:

Proof:

E(E(Y j X)) = E(m(X))

Law of Iterated Expectations:

where m(x) = E(Y j X = x) : Thus h(X) = g(X)m(X), which is the same as E(g(X)Y j X) = g (X) E(Y j X) :

B.8 Transformations

Suppose that X 2 Rk with continuous distribution function FX(x) and density fX(x): Let Y = g(X) where g(x) : Rk ! Rk is one-to-one, di¤erentiable, and invertible. Let h(y) denote the

Consider the univariate case k = 1: If g(x) is an increasing function, then g(X) Y if and only

This is known as the change-of-variables formula. This same formula (B.11) holds for k > 1; but its justi…cation requires deeper results from analysis.

As one example, take the case X U[0; 1] and Y = log(X). Here, g(x) = log(x) and h(y) = exp( y) so the Jacobian is J(y) = exp(y): As the range of X is [0; 1]; that for Y is [0,1): Since fX (x) = 1 for 0 x 1 (B.11) shows that

fY (y) = exp( y); 0 y 1;

an exponential density.