b

n

b

Xi

Rtr(S) =

=1

(Yi(S) Yi)2

R2

rss(S) = 1

Rtr(S) = 1

in=1(Yi(S)

)2

R2(S) = 1

Y

P in=1(Yi

tss

tss

Y

)2

b

b

P

h

i

E Rbtr(S) < R(S)

n

bias(Rtr(S)) = E hRtr(S)i

R(S) = 2 i=1 Cov hYi; Yii

b

b

X

b

m

b

Xi

b

i

jf

gj

R

V

(S) =

i

m =

4

2

(Y

(S)

Y )2

validation data

; often n or

n

=1

RCV (S) =

(Yi Y(i))2 =

2

n

n

1i Uii(S) !

X

Xi

Y

Yi(S)

b

b

b

i=1

=1

KDE

f

(x) =

1

n

1

K

x Xi

n

n i=1 h

h

b

1

X

1

R(f; fn)

(h K)4 Z (f00(x))2 dx +

Z K2(x) dx

4

nh

b

c 2=5c2 1=5c3 1=5

2

; c2 = Z

2

= Z

2

h =

1

c1 = K

K

(x) dx; c3

(f00(x))

dx

n1=5

R (f; fn) = n4=5

c4

= 4( K2 )2=5 Z K2(x) dx

4=5

Z (f00)2 dx

1=5

b

c4

5

|

}

C

(K)

{z

k-nearest Neighbor Estimator

1

X

where Nk(x) = fk values of x1; : : : ; xn closest to xg

b

Yi

r(x) = k

i:xi2Nk(x)

Nadaraya-Watson Kernel Estimator

b

n

K

x hxi

Xi

r(x) =

wi(x)Yi

=1

K

wi(x) =

2 [0; 1]

n

x xj

2

4j=1

h

4

hP

Z

x2K2(x) dx Z r00

f (x)

dx

R(rn; r)

(x) + 2r0

(x)

0

4

f(x)

b

2

K2(x) dx

+ Z

c1

Rnhf(x)

dx

h

n1=5

R (r

; r)

c2

n4=5

bn

Epanechnikov Kernel

(0

otherwisej j

K(x) =

3

x < p

4p

5(1 x2=5)

5

Cross-validation estimate of E [J(h)]

JCV (h) = Z

fn2

2 n

1

n n

(x) dx n i=1 f( i)(Xi) hn2

i=1 j=1 K

b

b

X b

XX

n

n

JCV (h) = (Yi r( i)(xi))2 =

X

(Yi

r(xi))2

2

Xi

K(0)

b

1

b

n

K

x

xj

!

b

i=1

j=1

h

=1

P

h

nh

1

J

Xi Xj

+

2

K(0)

Approximation

j=1

i=1

Multivariate regression

Y = +

where i = i and

=

0 0(...x1)

...

J (...x1)

1

B 0(xn)

J (xn)C

@

A