Probability theory
Law of large numbers

Common intuition suggests that if a fair coin is tossed many times, then roughly half of the time it will turn up heads, and the other half it will turn up tails. Furthermore, the more often the coin is tossed, the more likely it should be that the ratio of the number of heads to the number of tails will approach unity. Modern probability theory provides a formal version of this intuitive idea, known as the law of large numbers. This law is remarkable because it is not assumed in the foundations of probability theory, but instead emerges from these foundations as a theorem. Since it links theoretically derived probabilities to their actual frequency of occurrence in the real world, the law of large numbers is considered as a pillar in the history of statistical theory and has had widespread influence.

The law of large numbers (LLN) states that the sample average

of a sequence of independent and identically distributed random variables X k {\displaystyle X_{k}} converges towards their common expectation (expected value) μ {\displaystyle \mu }, provided that the expectation of | X k | {\displaystyle |X_{k}|} is finite.

It is in the different forms of convergence of random variables that separates the weak and the strong law of large numbers

It follows from the LLN that if an event of probability p is observed repeatedly during independent experiments, the ratio of the observed frequency of that event to the total number of repetitions converges towards p.

For example, if Y 1, Y 2,... {\displaystyle Y_{1},Y_{2},...\,} are independent Bernoulli random variables taking values 1 with probability p and 0 with probability 1- p, then E (Y i) = p {\displaystyle {\textrm {E}}(Y_{i})=p} for all i, so that Y ¯ n {\displaystyle {\bar {Y}}_{n}} converges to p almost surely.
keywords:frequency