Professional Documents
Culture Documents
Theory
Bayesian probability
Probability interpretations
Bayes' theorem
Bayes' rule
Bayes factor
Bayesian inference
Bayesian network
Likelihood
Posterior predictive
Hyperparameter
Posterior
Conjugate prior
Prior
Hyperprior
Principle of indifference
Cromwell's rule
Credible interval
Techniques
Bayesian estimator
Uses
Binary classification
Naive Bayes classifier
Statistics portal
In probability theory and applications, Bayes' rule relates the odds of event
to
event
, before and after conditioning on event . The relationship is expressed in
terms of the Bayes factor, . Bayes' rule is derived from and closely related to Bayes'
theorem. Bayes' rule may be preferred to Bayes' theorem when the relative probability
(that is, the odds) of two events matters, but the individual probabilities do not. This is
because in Bayes' rule,
is eliminated and need not be calculated (see
Derivation). It is commonly used in science and engineering, notably for model
selection.
Under the frequentist interpretation of probability, Bayes' rule is a general relationship
between
and
same event space. In this case,
odds.
Under the Bayesian interpretation of probability, Bayes' rule relates the odds on
probability models
and
before and after evidence is observed. In this case,
represents the impact of the evidence on the odds. This is a form of Bayesian
inference - the quantity
is called the prior odds, and
the posterior odds. By analogy to the prior and posterior probability terms in Bayes'
theorem, Bayes' rule can be seen as Bayes' theorem in odds form. For more detail on
the application of Bayes' rule under the Bayesian interpretation of probability, see
Bayesian model selection.
Contents
1 The rule
o 1.1 Single event
o 1.2 Multiple events
2 Derivation
3 Examples
o 3.1 Frequentist example
o 3.2 Model selection
4 External links
The rule
Single event
Given events
,
and , Bayes' rule states that the conditional odds of
given are equal to the marginal odds of
multiplied by the Bayes factor
where
and
Multiple events
Bayes' rule may be conditioned on an arbitrary number of events. For two events
and ,
where
Derivation
Consider two instances of Bayes' theorem:
Now defining
this implies
A similar derivation applies for conditioning on multiple events, using the appropriate
extension of Bayes' theorem
Examples
Frequentist example
Consider the drug testing example in the article on Bayes' theorem.
The same results may be obtained using Bayes' rule. The prior odds on an individual
being a drug-user are 199 to 1 against, as
and
. The
Model selection