3.1 Transformations
Transformation function
|
|
|
Z = '(X) |
|
|
|
|
|
|
|
|
Discrete |
|
|
|
|
|
|
|
|
|
2X |
|
|
|
|
|
|
|
|
|
||||
fZ(z) = P ['(X) = z] = P [fx : '(x) = zg] = P X 2 ' 1(z) = |
|
|
f(x) |
||||||||
|
|
|
|
|
|
|
|
|
x |
|
' 1(z) |
Continuous |
|
|
|
|
|
|
|
|
|
|
|
FZ(z) = P ['(X) z] = ZAz f(x) dx with Az = fx : '(x) zg |
|||||||||||
Special case if ' strictly monotone |
(z) = fX(x) |
dz |
|
|
|
|
|
||||
fZ(z) = fX(' 1(z)) |
dz ' 1 |
= fX(x) |
J |
|
|||||||
|
|
d |
|
|
dx |
|
1 |
j |
|||
|
|
|
|
|
|
j |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The Rule of the Lazy Statistician
Z
E [Z] = '(x) dFX(x)
ZZ
|
E [IA(x)] = IA(x) dFX(x) = |
A dFX(x) = P [X 2 A] |
||||||||
Convolution |
|
|
|
|
|
|
Z0 |
|
||
|
|
|
|
1 |
|
|
X;Y |
0 |
z |
|
Z := X + Y |
|
fZ(z) = Z 1 fX;Y (x; z x) dx |
= |
|
fX;Y (x; z x) dx |
|||||
Z := jX Y j |
fZ(z) = 2 Z0 |
1 fX;Y (x; z + x) dx |
|
|
|
|||||
|
X |
|
1 |
|
|
1 |
|
|
|
|
Z := |
|
|
fZ(z) = Z 1 jxjfX;Y (x; xz) dx ??= |
Z 1 xfx(x)fX(x)fY (xz) dx |
||||||
Y |
||||||||||
4Expectation
De nition and properties
8
X
> xfX(x)
>
>
Z> x
<
E [X] = X = x dFX(x) =
Z
>
>
>
> xf (x)
: X
P [X = c] = 1 =) E [c] = c
E [cX] = c E [X]
E [X + Y ] = E [X] + E [Y ]
Xdiscrete
Xcontinuous
E [XY ] = ZX;Y |
xyfX;Y (x; y) dFX(x) dFY (y) |
|||
E ['(Y )] 6= '(E [X]) (cf. Jensen inequality) |
||||
P [X Y ] = 0 =) E [X] E [Y ] ^ P [X = Y ] = 1 =) E [X] = E [Y ] |
||||
1 |
|
|
|
|
X |
|
|
|
|
E [X] = P [X x] |
|
|||
x=1 |
|
|
|
|
Sample mean |
|
|
|
n |
|
|
|
1 |
|
|
|
Xn = n |
Xi |
|
|
|
Xi |
||
|
|
|
|
=1 |
Conditional expectation |
|
|
|
|
E [Y j X = x] = Z |
yf(y j x) dy |
|
||
E [X] = E [E [X j Y ]] |
|
|||
Z 1
E['(X; Y ) j X = x] = '(x; y)fY jX(y j x) dx
1
Z1
E ['(Y; Z) j X = x] = '(y; z)f(Y;Z)jX(y; z j x) dy dz
1
E [Y + Z j X] = E [Y j X] + E [Z j X]
E ['(X)Y j X] = '(X)E [Y j X]
E[Y j X] = c =) Cov [X; Y ] = 0
5Variance
De nition and properties
V |
[X] = 2 |
= E (X |
E |
[X])2 |
|
= E X2 |
|
|
E [X]2 |
||
" |
n |
X |
n |
|
|
|
|||||
V |
X |
Xi# |
= V [Xi] + 2 |
|
Cov [Xi; Yj] |
|
|||||
|
|
|
X |
|
X |
|
|
|
|
||
|
|
i=1 |
|
i=1 |
|
i6=j |
|
|
|
|
|
" n # n
XX
V |
Xi = V [Xi] if Xi ?? Xj |
i=1 |
i=1 |
Standard deviation
p
sd[X] = V [X] = X
Covariance
Cov [X; Y ] = E [(X E [X])(Y E [Y ])] = E [XY ] E [X] E [Y ]
Cov [X; a] = 0
Cov [X; X] = V [X]
Cov [X; Y ] = Cov [Y; X]
Cov [aX; bY ] = abCov [X; Y ]
7
Cov [X + a; Y + b] = Cov [X; Y ]
|
2 |
|
|
3 |
|
|
|
n |
m |
|
|
n |
m |
Cov |
X |
Xj |
Yj |
|
XX |
|
|
4 |
|
|
5 |
|
|
|
i=1 |
=1 |
|
|
i=1 j=1 |
|
Correlation
Cov [X; Y ][X; Y ] = p
V [X] V [Y ]
Independence
X ?? Y =) [X; Y ] = 0 () Cov [X; Y ] = 0 () E [XY ] = E [X] E [Y ]
Sample variance
|
|
|
|
|
|
1 |
|
|
|
n |
|
|
|
|
|
|
|
|
|
|
Xi |
|
|
|
|
|
|
|
|
|
|
|
|
||
|
|
|
2 |
|
|
|
|
|
|
|
2 |
|
|
S |
|
= |
n 1 |
(Xi Xn) |
|
||||
|
|
|
|
|
|
|
|
|
|
=1 |
|
Conditional variance |
|
|
|
|
|
|
|
|
|
|
|
V [Y j X] = E (Y E [Y j X])2 j X = E Y 2 j X E [Y j X]2 |
|||||||||||
Y |
j |
X]] + |
|
[ |
E |
[Y |
j |
X]] |
|
||
V [Y ] = E [V [ |
|
V |
|
|
|
|
|||||
6Inequalities
Cauchy-Schwarz |
|
|
|
|
|
|
|
|
||
Markov |
E [XY ]2 E X2 E Y 2 |
|
||||||||
|
P ['(X) t] |
E ['(X)] |
|
|||||||
|
|
|
t |
|
|
|
|
|||
Chebyshev |
|
|
|
|
V [X] |
|
||||
|
P [jX E [X]j t] |
|
||||||||
|
|
|
t2 |
|
|
|||||
Chernoff |
P [X (1 + ) ] |
|
|
|
|
|
|
|
||
|
|
|
|
|
|
|||||
|
e |
> 1 |
||||||||
|
(1 + )1+ |
|||||||||
Jensen |
E ['(X)] '(E [X]) |
|
|
|
|
|
|
|
||
|
' convex |
|||||||||
7Distribution Relationships
Binomial
n
X
Xi Bern (p) =) Xi Bin (n; p)
i=1
X Bin (n; p) ; Y Bin (m; p) =) X + Y Bin (n + m; p)
limn!1 Bin (n; p) = Po (np) (n large, p small)
limn!1 Bin (n; p) = N (np; np(1 p)) (n large, p far from 0 and 1)
Negative Binomial |
|
X NBin (1; p) = |
Geo (p) |
r |
|
X NBin (r; p) = |
Pi=1 Geo (p) |
PP
Xi NBin (ri; p) =) Xi NBin ( ri; p)
X NBin (r; p) : Y Bin (s + r; p) =) P [X s] = P [Y r]
Poisson
|
|
|
|
|
|
|
|
|
n |
|
|
|
n |
! |
|
|
|
|
|
|
Xi Po ( i) ^ Xi ?? Xj =) |
X |
Xi Po |
X |
i |
|
|
|
|
|
|
||||||||||
|
|
|
|
|
|
|
|
|
i=1 |
n |
|
|
i=1 |
0 n |
|
|
|
|
|
1 |
|
Xi |
|
Po ( i) |
^ |
Xi |
?? |
Xj |
= |
Xi |
Xj |
|
Bin |
Xj; |
|
i |
|
|
|||
|
n |
|
|
|||||||||||||||||
|
|
|
|
) |
|
j=1 |
|
|
j=1 |
|
|
j=1 |
|
j |
||||||
|
|
|
|
|
|
|
|
|
|
X |
|
|
|
X |
|
P |
|
|
|
A |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@ |
|
|
|
|
||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Exponential |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|||
|
|
|
|
|
|
|
|
|
n |
|
|
|
|
|
|
|
|
|
|
|
Xi |
Exp ( ) ^ Xi ?? Xj |
=) XXi |
Gamma (n; ) |
|
|
|
|
|
||||||||||||
i=1
Memoryless property: P [X > x + y j X > y] = P [X > x] Normal
X |
N |
|
|
2 |
|
|
|
|
X |
|
|
Z |
|
|
a + b; a2 2 |
|
|
|
|||||
; 2 |
|
|
Z = aX + b =N |
|
|
|
|
|
|||||||||||||||
|
X |
|
; |
|
=) |
|
2 |
; 2 |
|
(0; 1) |
|
|
|
|
1 |
|
|
|
|||||
|
|
N |
|
1 1 |
^ |
|
|
|
|
= X |
|
|
|
2 1 |
2 |
|
|||||||
|
Xi |
|
|
i; i |
|
|
= |
i i |
2 |
|
|
i i; |
|
i |
i |
|
|
|
+ 2 |
|
|||
|
X |
N |
; 2 |
|
^ |
Y |
N |
|
|
) |
+ Y |
N |
+ ; 2 |
|
|||||||||
N |
|
|
2 |
|
|
|
) P |
X |
N |
P P |
2 |
|
|
|
|
|
|||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
P [a < X b] = |
b |
|
a |
|
|
|
|||||
|
|
00(x) = (x2 |
|
1) (x) |
||||||||
( |
|
x) = 1 |
|
(x) |
0 |
(x) = |
|
x (x) |
|
|||
|
|
|
|
|
|
|
|
|
||||
Upper quantile of N (0; 1): z = 1(1 ) Gamma
X Gamma ( ; ) () X= Gamma ( ; 1)
Gamma ( ; ) Pi=1 Exp ( )
P P
Xi Gamma ( i; ) ^ Xi ?? Xj =) i i; )
( ) = Z 1 x 1e x dx0
Beta |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
|
|
|
|
|
|
( + ) |
|
|
|
||
|
|
|
x 1(1 x) 1 = |
|
x 1(1 x) 1 |
||||||||
B( ; ) |
( ) ( ) |
||||||||||||
|
|
Xk |
|
= |
B( + k; ) |
= |
|
+ k 1 |
|
Xk 1 |
|
||
|
|
B( ; ) |
|
+ + k 1E |
|||||||||
E |
|
|
|
||||||||||
Beta (1; 1) Unif (0; 1)
8