quality:correction, linguistic:voice_change
Proof of the strong law

We give a relatively simple proof of the strong law under the assumptions that the X i {\displaystyle X_{i}} are iid, E =: μ < ∞ {\displaystyle {\mathbb {E} }=:\mu <\infty }, Var (X i) = σ 2 < ∞ {\displaystyle \operatorname {Var} (X_{i})=\sigma ^{2}<\infty }, and E =: τ < ∞ {\displaystyle {\mathbb {E} }=:\tau <\infty }.

Let us first note that without loss of generality we can assume that μ = 0 {\displaystyle \mu =0} by centering. In this case, the strong law says that

Pr (lim n → ∞ X ¯ n = 0) = 1, {\displaystyle \Pr \!\left(\lim _{n\to \infty }{\overline {X}}_{n}=0\right)=1,} or Pr (ω: lim n → ∞ S n (ω) n = 0) = 1. {\displaystyle \Pr \left(\omega :\lim _{n\to \infty }{\frac {S_{n}(\omega)}{n}}=0\right)=1.} It is equivalent to show that Pr (ω: lim n → ∞ S n (ω) n ≠ 0) = 0, {\displaystyle \Pr \left(\omega :\lim _{n\to \infty }{\frac {S_{n}(\omega)}{n}}\neq 0\right)=0,} Note that lim n → ∞ S n (ω) n ≠ 0 ⟺ ∃ ϵ > 0, | S n (ω) n | ≥ ϵ infinitely often, {\displaystyle \lim _{n\to \infty }{\frac {S_{n}(\omega)}{n}}\neq 0\iff \exists \epsilon >0,\left|{\frac {S_{n}(\omega)}{n}}\right|\geq \epsilon \ {\mbox{infinitely often}},} and thus to prove the strong law we need to show that for every ϵ > 0 {\displaystyle \epsilon >0}, we have Pr (ω: | S n (ω) | ≥ n ϵ infinitely often) = 0. {\displaystyle \Pr \left(\omega :|S_{n}(\omega)|\geq n\epsilon {\mbox{ infinitely often}}\right)=0.} Define the events A n = { ω: | S n | ≥ n ϵ } {\displaystyle A_{n}=\{\omega :|S_{n}|\geq n\epsilon \}}, and if we can show that ∑ n = 1 ∞ Pr (A n) < ∞, {\displaystyle \sum _{n=1}^{\infty }\Pr(A_{n})<\infty,} then the Borel-Cantelli Lemma implies the result. So let us estimate Pr (A n) {\displaystyle \Pr(A_{n})}.

We compute E = E = E. {\displaystyle {\mathbb {E} }={\mathbb {E} }\left={\mathbb {E} }\left.} We first claim that every term of the form X i 3 X j, X i 2 X j X k, X i X j X k X l {\displaystyle X_{i}^{3}X_{j},X_{i}^{2}X_{j}X_{k},X_{i}X_{j}X_{k}X_{l}} where all subscripts are distinct, must have zero expectation. This is because E = E E {\displaystyle {\mathbb {E} }={\mathbb {E} }{\mathbb {E} }} by independence, and the last term is zero --- and similarly for the other terms. Therefore the only terms in the sum with nonzero expectation are E {\displaystyle {\mathbb {E} }} and E {\displaystyle {\mathbb {E} }}. Since the X i {\displaystyle X_{i}} are identically distributed, all of these are the same, and moreover E = (E) 2 {\displaystyle {\mathbb {E} }=({\mathbb {E} })^{2}}.

There are n {\displaystyle n} terms of the form E {\displaystyle {\mathbb {E} }} and 3 n (n − 1) {\displaystyle 3n(n-1)} terms of the form (E) 2 {\displaystyle ({\mathbb {E} })^{2}}, and so E = n τ + 3 n (n − 1) σ 4. {\displaystyle {\mathbb {E} }=n\tau +3n(n-1)\sigma ^{4}.} Note that the right-hand side is a quadratic polynomial in n {\displaystyle n}, and as such there exists a C > 0 {\displaystyle C>0} such that E ≤ C n 2 {\displaystyle {\mathbb {E} }\leq Cn^{2}} for n {\displaystyle n} sufficiently large. By Markov, Pr (| S n | ≥ n ϵ) ≤ 1 (n ϵ) 4 E ≤ C ϵ 4 n 2, {\displaystyle \Pr(|S_{n}|\geq n\epsilon)\leq {\frac {1}{(n\epsilon)^{4}}}{\mathbb {E} }\leq {\frac {C}{\epsilon ^{4}n^{2}}},} for n {\displaystyle n} sufficiently large, and therefore this series is summable. Since this holds for any ϵ > 0 {\displaystyle \epsilon >0}, we have established the Strong LLN.

Another proof can be found in

For a proof without the added assumption of a finite fourth moment, see Section 22 of.