- •Distribution Overview
- •Discrete Distributions
- •Continuous Distributions
- •Probability Theory
- •Random Variables
- •Transformations
- •Expectation
- •Variance
- •Inequalities
- •Distribution Relationships
- •Probability and Moment Generating Functions
- •Multivariate Distributions
- •Standard Bivariate Normal
- •Bivariate Normal
- •Multivariate Normal
- •Convergence
- •Statistical Inference
- •Point Estimation
- •Empirical distribution
- •Statistical Functionals
- •Parametric Inference
- •Method of Moments
- •Maximum Likelihood
- •Delta Method
- •Multiparameter Models
- •Multiparameter delta method
- •Parametric Bootstrap
- •Hypothesis Testing
- •Bayesian Inference
- •Credible Intervals
- •Function of parameters
- •Priors
- •Conjugate Priors
- •Bayesian Testing
- •Exponential Family
- •Sampling Methods
- •The Bootstrap
- •Rejection Sampling
- •Importance Sampling
- •Decision Theory
- •Risk
- •Admissibility
- •Bayes Rule
- •Minimax Rules
- •Linear Regression
- •Simple Linear Regression
- •Prediction
- •Multiple Regression
- •Model Selection
- •Non-parametric Function Estimation
- •Density Estimation
- •Histograms
- •Kernel Density Estimator (KDE)
- •Smoothing Using Orthogonal Functions
- •Stochastic Processes
- •Markov Chains
- •Poisson Processes
- •Time Series
- •Stationary Time Series
- •Estimation of Correlation
- •Detrending
- •ARIMA models
- •Causality and Invertibility
- •Spectral Analysis
- •Math
- •Gamma Function
- •Beta Function
- •Series
- •Combinatorics
14.3Priors
Choice
Subjective bayesianism.
Objective bayesianism.
Robust bayesianism.
Types
Flat: f( ) / constant
R1
Proper: 1 f( ) d = 1
R1
Improper: 1 f( ) d = 1
Jeffrey's prior (transformation-invariant):
pp
f( ) / I( ) f( ) / det(I( ))
Conjugate: f( ) and f( j xn) belong to the same parametric family
14.3.1Conjugate Priors
Discrete likelihood
Likelihood |
Conjugate prior |
Posterior hyperparameters |
|||
|
|
|
|
|
|
|
|
n |
|
|
n |
Bern (p) |
Beta ( ; ) |
+ |
xi; + n |
xi |
|
|
|
i=1 |
|
n |
i=1 |
|
|
n |
|
n |
|
|
|
X |
|
|
X |
Bin (p) |
Beta ( ; ) |
+ |
xi; + |
Ni xi |
|
|
|
=1 |
n |
i=1 |
i=1 |
|
|
Xi |
X |
X |
|
NBin (p) |
Beta ( ; ) |
+ rn; + |
xi |
|
|
|
|
n |
=1 |
|
|
|
|
Xi |
|
||
Po ( ) |
Gamma ( ; ) |
+ |
xi; + n |
|
|
|
|
i=1 |
|
|
|
|
|
n |
|
|
|
|
|
X |
|
|
|
Multinomial(p) |
Dir ( ) |
+ |
x(i) |
|
|
|
|
=1 |
n |
|
|
|
|
Xi |
|
|
|
Geo (p) |
Beta ( ; ) |
+ n; + |
xi |
|
|
|
|
|
=1 |
|
|
|
|
|
Xi |
|
|
Continuous likelihood (subscript c denotes constant)
Likelihood |
Conjugate prior |
Posterior hyperparameters |
|
|
|
|
|
|
|||||||||||||||||||||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||
|
|
|
|
|
|
m |
|
|
|
|
|
|
|
(n) |
|
n |
|
|
|
|
|
|
|
|
|
|
|
|
|||||||
Unif (0; ) |
Pareto(x |
|
; k) |
max |
x |
; xm |
|
|
; k + n |
|
|
|
|
|
|
|
|||||||||||||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Xi |
xi |
|
|
|
|
|
|
|
|
|
|
|
|
||||
Exp ( ) |
|
|
Gamma ( ; ) |
+ n; + |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||||||||||||||
|
|
|
|
|
|
|
|
|
02 |
|
P |
|
=1 |
|
|
|
|
|
|
c2 |
|
|
|
||||||||||||
N |
|
|
|
N |
|
|
|
|
|
n |
|
|
|
02 |
|
|
|
||||||||||||||||||
|
|
|
|
|
|
|
|
c2 |
|
|
|
|
|
||||||||||||||||||||||
|
; 2 |
|
|
; 2 |
|
|
|
0 |
+ |
|
|
i=1 xi |
|
= |
1 |
+ |
n |
, |
|
|
|
||||||||||||||
|
1 |
|
n |
|
|
|
|
|
|
|
|
||||||||||||||||||||||||
c |
|
|
|
|
|
|
|
|
|
1 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||||||||||
|
|
|
|
0 0 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
|
|
|
|
|
|
|
|
|
|
+ |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||
|
|
|
|
|
|
|
|
|
02 |
c2 |
|
|
|
in=1(xi )2 |
|
|
|
|
|
||||||||||||||||
N |
c; 2 |
|
Scaled Inverse Chi- |
+ n, |
02 + |
|
|
|
|
|
|
|
|||||||||||||||||||||||
|
|
|
|
|
|
|
|
|
|
|
|
||||||||||||||||||||||||
|
|
|
|
|
|
|
|
0 |
|
|
|
|
|
|
|
|
|
|
|
|
|
+ n |
|
|
|
|
|
|
|
|
|
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||||
N |
; 2 |
|
Normal- |
|
|
|
+ nx |
, |
|
|
|
|
+ n, |
|
|
+ |
n |
, |
|||||||||||||||||
|
Inverse |
|
|
|
|
|
|
|
|
|
|||||||||||||||||||||||||
|
scaled |
|
|
|
|
1 |
|
n |
|
|
|
|
|
|
|
|
(x |
|
)2 |
2 |
|
||||||||||||||
|
|
|
|
|
|
|
|
|
|
|
|
+ n |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||
|
|
|
|
Gamma( ; ; ; ) |
|
|
|
|
|
|
Xi |
|
|
|
|
x)2 + |
|
|
|
|
|
|
|
|
|
|
|||||||||
|
|
|
|
+ |
|
|
|
(x |
|
|
|
2(n + ) |
|
|
|
||||||||||||||||||||
|
|
|
|
|
|
|
|
|
|
|
|
2 |
|
=1 |
|
i |
|
|
|
|
|
|
|
|
|||||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
MVN( ; c) |
MVN( 0; 0) |
|
1 + n 1 |
1 |
|
1 0 |
+ n 1x , |
||||||||||||||||||||||||||||
|
|
|
|
|
|
|
|
|
0 1 |
+ n c 1 1 |
|
|
|
|
|
|
|
|
|
||||||||||||||||
|
|
|
|
|
|
|
|
|
0 |
|
|
|
|
|
|
c |
|
|
|
|
0 |
|
|
|
|
|
|
|
|
|
|
||||
|
|
|
|
Wishart( ; ) |
|
|
|
|
|
|
|
|
|
n |
|
|
|
|
|
|
|
|
|
|
|
|
T |
||||||||
MVN( c; ) |
|
|
|
|
|
|
|
|
|
|
Xi |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||||||
Inverse- |
|
|
n + ; + (xi c)(xi c) |
|
|
|
|||||||||||||||||||||||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
=1 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
n |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Xi |
|
|
|
|
xi |
|
|
|
|
|
|
|
|
|
|
||
Pareto(xmc ; k) |
Gamma ( ; ) |
+ n; + |
=1 |
log |
xmc |
|
|
|
|
|
|
|
|||||||||||||||||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||||
|
|
|
|
|
|
|
|
|
x0; k0 kn |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||||||||
Pareto(xm; kc) |
Pareto(x0; k0) |
where |
|
k0 > kn |
|
|
|
|
|
|
|||||||||||||||||||||||||
|
|
|
|
n |
|
|
|
|
|
|
|
||||||||||||||||||||||||
Gamma ( c; ) |
Gamma ( 0; 0) |
0 + n c; 0 + |
Xi |
|
|
|
|
|
|
|
|
|
|
||||||||||||||||||||||
|
|
|
xi |
|
|
|
|
|
|
|
|
|
|
||||||||||||||||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
=1 |
|
|
|
|
|
|
|
|
|
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
14.4Bayesian Testing
If H0 : 2 0: |
Prior probability P [H0] = Z 0 |
|
|
||||
|
f( ) d |
||||||
Posterior probability P [H0 j xn] = Z 0 |
f( j xn) d |
||||||
Let H0; : : : ; HK 1 be K hypotheses. Suppose f( j Hk), |
|||||||
P |
[H |
k j |
xn] = |
f(xn j Hk)P [Hk] |
; |
||
|
|
K |
n |
[Hk] |
|
||
|
|
|
|
Pk=1 f(x |
j Hk)P |
15 |