Bayes' Theorem

There are those who might argue that the thinking behind this was less Thomas Bayes and more to do with the man who edited the work and published it for Bayes after Bayes' death. His name was Richard Price, and was an interesting chap in his own right, and was born in Llangeinor, South Wales.

Bayes' Theorem provides a way to apply quantitative reasoning to what we normally think of as the scientific method. When several alternative hypotheses compete for our belief, we test them by deducing consequences of each one, then conducting experimental tests to observe whether or not those consequences actually occur. If an hypothesis predicts that something should occur, and that thing does occur, it strengthens our belief in the truthfulness of the hypothesis. Conversely, an observation that contradicts the prediction would weaken (or destroy) our confidence in the hypothesis.

In many situations, the predictions involve probabilities. For instance, one hypothesis might predict that a certain outcome has X% chance of occurring, while a competing hypothesis might predict Y% chance of the same outcome. In these situations, the occurrence or non-occurrence of the outcome would shift our relative degree of believe from one hypothesis toward another. Bayes theorem provides a way to calculate these degree of belief adjustments.

In Bayes' Theorem terminology, we first construct a set of mutually-exclusive and all-inclusive hypothesis and spread our degree of belief among them by assigning a "prior probability" (number between 0 and 1) to each hypothesis. If we have no prior basis for assigning probabilities, we could just spread our "belief probability" evenly among the hypotheses.

Then we construct a list of possible observable outcomes. This list should also be mutually exclusive and all inclusive. For each hypothesis we calculate the "conditional probability" of each possible outcome. This is just the probability of observing each outcome if that particular hypothesis is true. For each hypothesis, the sum of the conditional probabilities for all the outcomes must add up to 1.

We then note which outcome actually occurred. Using Bayes' formula, we can then compute revised post hoc probabilities for the hypotheses.

Several good internet sites look at Bayes' Theorem, of which the following might be worth a visit:
Stanford encyclopaedia of Philosophy
Wikipedia