Professional Documents
Culture Documents
Bayesian Inference
4/10/09
Bayesian Inference
4/10/09
" = p(x # R | $ # %1 )
has no definite value. The best one can do is to evaluate
p(x " R | # ) = $ (# )
Bayesian Inference
!
!4/10/09
!
Bayesian Inference
4/10/09
Bayesian Inference
4/10/09
Bayesian Inference
4/10/09
Marginal Likelihood
Bayesian Inference
4/10/09
Bayesian Inference
4/10/09
Marginal Likelihood
Marginal Likelihood
p(H j | x) =
# p(" , H
j
# p(x | " , H
j
j=0
!
4/10/09
Bayesian Inference
!
p(H j | x) " [ $ p(x | # j , H j ) p(# j | H j )d# j ] p(H j ) = m(x | H j )p(H j )
Bayesian Inference
| x)d" j
p(x)
where
n
4/10/09
10
!
Bayesian Inference
4/10/09
11
Bayesian Inference
4/10/09
12
p(H 0 | x)
=
p(H 1 | x)
%"
0
%"
0
&
p(H 0 )
p(H 1 )
= (h + t +1)Chh+t (0.5)h+t
=
(n +1)Chn
with n = h + t
2n
!
Bayesian Inference
4/10/09
13
Bayesian Inference
4/10/09
p-values
p - value =
100
60
101 ! C
= 1.095
2100
In other words, these data (slightly) favor the null
hypothesis that the coin is fair!
Surprising to a frequentist since the one-sided
p-value (tail area for obtaining 60 or more heads on
100 tosses given a fair coin) is 0.028 which would
reject the null in an #=0.05 level test
Bayesian Inference
4/10/09
14
$ p(r | "
)dr
r0
% r0
p - value =
$ p(r | "
0 )dr +
%#
$ p(r | "
)dr
r0
!
15
Bayesian Inference
4/10/09
16
Bayesian Inference
4/10/09
17
Bayesian Inference
4/10/09
18
MCMC Simulation
Bayesian Inference
4/10/09
19
Bayesian Inference
4/10/09
20
MCMC Simulation
MCMC Simulation
Bayesian Inference
4/10/09
21
Bayesian Inference
MCMC Simulation
! m|n =
Bayesian Inference
4/10/09
22
MCMC Simulation
! m|n =
4/10/09
23
Bayesian Inference
4/10/09
24
MCMC Simulation
MCMC Simulation
! m|n =
!
Bayesian Inference
4/10/09
25
Bayesian Inference
p-values
4/10/09
26
p-values
Bayesian Inference
4/10/09
27
Bayesian Inference
4/10/09
28
p-values
p-values
p(e)
0
Bayesian Inference
x!
4/10/09
29
Bayesian Inference
p-values
4/10/09
30
p-values
Bayesian Inference
4/10/09
31
Bayesian Inference
4/10/09
32
Falsification
Bayesian Epistemology
Bayesian Inference
4/10/09
33
Bayesian Inference
Ockhams Razor
4/10/09
34
Ockhams Razor
Bayesian Inference
4/10/09
35
Bayesian Inference
4/10/09
36
Simplicity
Plagiarism
Bayesian Inference
4/10/09
37
Bayesian Inference
Plagiarism
4/10/09
39
Evolutionary Biology
Bayesian Inference
4/10/09
40
Bayesian Inference
4/10/09
42
Bayesian Inference
4/10/09
p(a | aE ) =
Bayesian Inference
4/10/09
4/10/09
where
DE =
1
$ a2 '
p(aF | F) =
exp # F2
2!"
%& 2" ()
Bayesian Inference
44
45
a # aE
a
"
= #0.69, DF = = 20.8, " = = 25.0
*
*
*
Bayesian Inference
4/10/09
47
Whats Happening
Examples
The Ockham factor arises from the fact that F spreads its
bets over a much larger portion of parameter space than
does E. Essentially E puts all its money on aE, a precise
value, spread out only by the error of observation (. On
the other hand, F spreads its bets out over a range that is
25 times bigger, and hence most of its probability is
wasted covering regions of the parameter space that
were not observed.
E makes a sharp prediction, F a fuzzy one
When the data come out even moderately close to where E
predicts it will, E is rewarded for the risk it took by getting
a larger share of the posterior probability
The factor of about 25 is just the dilution in probability
that F must sacrifice in order to fit a larger range of data
Bayesian Inference
4/10/09
) D2 ! ' 2 $ ,
! E $ p(a | E)
&
F# & =
= 1 + ' 2 exp ( E #
+* 2 " 1 + ' 2 % ." F % p(a | F)
48
Bayesian Inference
Examples
53
Examples
1.2
0.8
Value
4/10/09
0.6
0.4
0.2
0
0
Ockham
factor
taubar
Poster ior Pr obability
Bayesian Inference
4/10/09
Evidence
factor
Bayes Factor
54
Bayesian Inference
4/10/09
55
Examples
Examples
From this we see that for a given DE, the more vague the
prior on the alternative (measured by ! ), the larger the
Bayes factor in favor of the sharp prediction of the null
Theories with great predictive power (sharp predictions)
are favored over those with vague predictions
Bayesian Inference
n! 2
= O( n) for large n
"2
Thus, asymptotically we have for large n
1+! 2 = 1+
) D2 ,
! E $ p(a | E)
F# & =
= ' exp ( E
+* 2 ." F % p(a | F)
Ockham
factor
"E %
" ( % + z2.
F $ ' ~ n $ ' exp-* 0
#) & , 2 /
#F&
where z=DE is the z-score, or standardized variable DE
Evidence
factor
4/10/09
56
Bayesian Inference
Examples
4/10/09
57
Jeffreys-Lindley Paradox
Bayesian Inference
4/10/09
$ z2 '
B ! n exp " #
%& 2 ()
Increases
with n
58
Fixed
Bayesian Inference
4/10/09
61
Jeffreys-Lindley Paradox
Jeffreys-Lindley Paradox
Bayesian Inference
4/10/09
62
Bayesian Inference
Jeffreys-Lindley Paradox
4/10/09
63
Jeffreys-Lindley Paradox
Bayesian Inference
4/10/09
64
Bayesian Inference
4/10/09
65
Jeffreys-Lindley Paradox
Jeffreys-Lindley Paradox
Bayesian Inference
4/10/09
66
Bayesian Inference
4/10/09
67
Bayesian Inference
4/10/09
68
Bayesian Inference
4/10/09
69
Bayesian Inference
4/10/09
70
Bayesian Inference
4/10/09
71
Bayesian Inference
4/10/09
72
Bayesian Inference
4/10/09
73
Frequentist Work-arounds
Bayesian Inference
4/10/09
74
Bayesian Inference
4/10/09
75
Goods Device
Bayesian Inference
4/10/09
76
Bayesian Inference
4/10/09
77
Example
Example
Bayesian Inference
4/10/09
obtain
%
(1/ 2
n
p(data | H 0 ) " '
* exp(nH ( f , p))
&2#r(n $ r))
" 1 ! f %'
H( f , p) = ! f ln( f / p) ! (1 ! f )ln $
#1 ! p&
78
Bayesian Inference
Example
4/10/09
79
Example
Bayesian Inference
4/10/09
80
Bayesian Inference
4/10/09
81
Example
Example
Bayesian Inference
4/10/09
82
Bayesian Inference
Example
4/10/09
83
Example
Bayesian Inference
4/10/09
84
Bayesian Inference
4/10/09
92
Making Decisions
Making Decisions
Bayesian Inference
4/10/09
93
Bayesian Inference
Making Decisions
4/10/09
94
Making Decisions
Bayesian Inference
4/10/09
95
Bayesian Inference
4/10/09
96
Making Decisions
Making Decisions
Bayesian Inference
4/10/09
97
Making Decisions
Bayesian Inference
4/10/09
99
Bayesian Inference
4/10/09
98