Bayesian inference
General formulation

Suppose a process is generating independent and identically distributed events E n, n = 1, 2, 3, … {\displaystyle E_{n},\ n=1,2,3,\ldots }, but the probability distribution is unknown. Let the event space Ω {\displaystyle \Omega } represent the current state of belief for this process. Each model is represented by event M m {\displaystyle M_{m}}. The conditional probabilities P (E n ∣ M m) {\displaystyle P(E_{n}\mid M_{m})} are specified to define the models. P (M m) {\displaystyle P(M_{m})} is the degree of belief in M m {\displaystyle M_{m}}. Before the first inference step, { P (M m) } {\displaystyle \{P(M_{m})\}} is a set of initial prior probabilities. These must sum to 1, but are otherwise arbitrary.

Suppose that the process is observed to generate E ∈ { E n } {\displaystyle E\in \{E_{n}\}}. For each M ∈ { M m } {\displaystyle M\in \{M_{m}\}}, the prior P (M) {\displaystyle P(M)} is updated to the posterior P (M ∣ E) {\displaystyle P(M\mid E)}. From Bayes' theorem:

P (M ∣ E) = P (E ∣ M) ∑ m P (E ∣ M m) P (M m) ⋅ P (M). {\displaystyle P(M\mid E)={\frac {P(E\mid M)}{\sum _{m}{P(E\mid M_{m})P(M_{m})}}}\cdot P(M).}

Upon observation of further evidence, this procedure may be repeated.
keywords:letter_frequency