Bayes rule (or Bayes estimator)

r(f; ) = inf

r(f; )

b

j

e

(x) = inf r(e

R

b

x)

8

)

j

x)f(x) dx

x = r(f; ) = r(

b

b

b

sup R( ; )

R(b) =

R(a) = sup R( ; a)

b

e

e

Model

E [ i j Xi] = 0; V [ i j Xi] = 2

Yi = 0 + 1Xi + i

Fitted line

b

Predicted ( tted) values

r(x) = b0 + b1x

Residuals

Ybi = r(Xi)

b

+ 1Xi

^i

= Yi Yi = Yi 0

b

b

b

Least square estimates

T

= (b0; b1)

T

min rss

b

:

b0;b1

0 = Yn

1Xn

=

n

b1

bn

P

n

Xn)2

P

n

i=1 Xi2

i=1

(Xi

Yn)

nX2

=

i=1(Xi

Xn)(Yi

i=1 XiYi

nXY

P

P

b

E h j Xni =

0

1

b

2

n 1

n Xi2

j

n

Xn

nsX2

Xn

1

hb

i

P

n=1 Xi2

V X =

i=1

se( 0) = sXpnr

P

i n

b

b

b

se( 1) =

sXbp

n

b

bn

1

n

where s2

= n 1

(X

X

n

)2

and 2 =

^2

(unbiased estimate).

i=1

n 2 i=1

X

i

i

Further

properties:

b

P

P

0 0

D

(0; 1)

and

1 1

D

(0; 1)

bse( 0)

! N

bse( 1)

! N

b

b

0

1

Approximate 1

bcon dence intervals for band :

b0 z =2se(b0)

and

b1 z =2se(b1)

0

1

b

1

1

6

b

j

j

=2

Wald test for

H :

= 0 vs. H : = 0: reject H0 if

W

> z where

R2

W = b1=se(b1).

b

n

n

R2 =

i=1

(Yi

Y )2

= 1

i=1 ^i2

= 1

rss

Pi=1

b

P

i P

tss

Y )2

Y )2

18

n

(Yi

n

(Yi

P

=1

n

n

n

iY

Y

Y

L =

f(Xi; Yi) =

fX(Xi) fY jX(Yi j Xi) = L1 L2

=1

i=1

i=1

n

iY

fX(Xi)

L1 =

L2

=1

fY jX(Yi j Xi) / n exp ( 2 2

Yi ( 0 1Xi)

)

=

n

1

2

i=1

i

Y

X

2 =

1 n

^i2

n

=1

b

Xi

Y = 0 + 1x

+ 2x Cov 0; 1

V Y = V 0

+ x2V 1

h

b

b b

h i

h

i

i

h i

Prediction interval

b

b

b

b

b

2

= 2

in=1(Xi X )2

+ 1

n

n

2

i

(Xi

X) j

b

P

b

P

Y = X +

where

0X...11 ...

X...1k 1 =

0 ...11

0 ...11

X =

=

BXn1

XnkC

B kC

B nC

Likelihood

@

A

@ A

@ A

L( ; ) = (2 2) n=2 exp

2 2 rss

1

N

rss = (y X )T (y X ) = kY X k2 =

Xi

(Yi xiT )2

=1

If the (k k) matrix XT X is invertible,

= (XT X) 1XT Y

V

Xnb

= 2(XT X) 1

h

j

i

2

(X

T

X)

1

b

;

Estimate regression function

b

N

k

r(x) =

jxj

b

=1

b

Unbiased estimate for 2

Xj

1

n

2

=

^i2

^ = X Y

mle

n

k

b

Xi

b

=1

= X

2 =

n k

2

b

b

n

R(S) =

mspe =

i=1 E

(Y

(S)

Y )2

i=1

i

bi

i

19