home | index | units | counting | geometry | algebra | trigonometry & functions | calculus analysis | sets & logic | number theory | recreational | misc | nomenclature & history | physics

# Bayesian StatisticsOn the Probability of Causes

When the facts change, I change my opinion.
What do you do, sir?

John Maynard Keynes   (1883-1946)

### Related Links (Outside this Site)

The Mathematics of Changing your Mind  by  John Allen Paulos  (2011)
Bayes' theorem  by  Kalid   ("Better Explained").
The Mascots of Bayesian Statistics  by  Rasmus Bååth  (2013-12-26).

Why Judea Pearl is only  half-Bayesian  by  Andrew Gelman.
Causality  by  Judea Pearl  (Second Edition, 2009)   [2000 review]

Wikipedia :   Bayesian Statistics

Videos :   How one equation changed the way I think  by  Julia Galef.
A visual guide to Bayesian thinking  by  Julia Galef  (2015-07-16).
Simpson's Paradox (14:05)  by  David Louapre  (in French, 2015-04-03).
The Bayesian Trap (10:36)  by  Derek Muller  (Veritasium, 2017-04-05).

## Bayesian Statistics  &  Probabilistic Inferences

(2013-10-30)   Bayesian Symmetry:  P(A,B)  =  P(B,A).   Is it obvious?
The relationship between joint probability and conditional probability.

In practice, the Bayesian formula  (which we shall presently introduce)  is often applied to an uncertain hypothesis A and a "test" B for it,  relating the  a priori  probability  P(A)  to the  a posteriori  probability  P(A|B).

However, both A and B can simply be considered to be events placed on the same footing, which do not play different mathematical rôles:

• P(A) denotes the probability of A.  P(B) is the probability of B.
• P(A,B) is the  joint probability  that A and B  both  occur.
• P(A|B) is the  conditional probability  of A when B does occur.

P(A|B) is read as the probability of  "A knowing B".  The following holds:

P(A|B) P(B)   =   P(A,B)   =   P(B,A)   =   P(B|A) P(A)

The fact that  togetherness  is symmetric  [i.e., P(A,B) = P(B,A) ]  is crucial in the above, which may serve as a proof of the following Bayes' formula of inference, upon which Bayesian statistics is based.  (The formula itself is due to Laplace,  as Bayes himself never bothered with a formal expression and neither did Price, who edited the work of Bayes posthumously).

Bayes' Theorem  (Bayes' Formula)
 P(A|B) = P(B|A)  P(A) P(B)

This is consistent with a classical description of reality where probabilities are expressed in terms of classical events which could be, among other possibilities, independent or mutually exclusive.

There are probabilistic systems which cannot be described in classical terms  (no two events are ever independent or mutually exclusive).  The  quantum universe  (which we live in)  is one such system, where the above foundational relation of Bayesian statistics is neither obvious nor true, as experimental violations of  Bell's inequality  demonstrate.

Bayes' theorem is just true in a self-consistent system where probability is a classical  measure  satisfying the following axiom of measure theory for the subsets of some  universal set  E of probability 1.

Thomas Bayes (1701-1761)   |   Richard Price (1723-1791)   |   Laplace (1749-1827)

Wikipedia :   Bayes' theorem   |   Bayesian probability   |   Bayesian inference

(2015-01-01)   The Bayesian Universe.
Probabilities quantify beliefs.

Wikipedia :   Sequential analysis   |   Decision theory

Ignoring true causes can make correlations meaningless.

Udny Yule (1871-1951)   |   Edward H. Simpson (1922-)

Machine Learning Methods (10:40)  by  Uwe Aickelin   (Computerphile, 2015-09-02).
Anti-learning (7:50)  by  Uwe Aickelin   (Computerphile, 2015-09-23).

Video :   Simpson's Paradox (4:39)  by  Henry Reich  (Minute Physics, 2017-10-24).

(2013-11-01)   Quantum Theory is not Bayesian.
Bayes' formula doesn't apply to the probabilities of quantum observations.

In quantum theory, events that can be placed on the same footing correspond to commuting observables.  Most pairs of observables do not commute and joint probabilities are strangely defined.

Wikipedia :   Quantum Bayesianism   |   Quantum information

(2013-10-30)   Is the human brain a Bayesian engine?

No, it's not.  At least not a perfect one.  If it was, then it wouldn't be capable of irrational decisions.

Arguably, the biology of the brain involves processes that entail  voting  and the associated irrational  nontransitivy  stemming from  Condorcet's paradox.

This is not to say, thankfully, that humans cannot consciously revise their beliefs or opinions when presented with new evidence.  It merely goes to say that it takes some effort to do so.  Just as it takes some effort to practice mathematics and obtain flawless results, in spite of our natural tendencies for intuition, preconceptions and mistakes.

(2015-03-30)   Causality
Post Hoc, Ergo Propter Hoc