linguistic:paraphrasing
With unknown mean and unknown variance

For a set of i.i.d. normally distributed data points X of size n where each individual point x follows x ∼ N (μ, σ 2) {\textstyle x\sim {\mathcal {N}}(\mu,\sigma ^{2})} with unknown mean μ and unknown variance σ, a combined (multivariate) conjugate prior is placed over the mean and variance, consisting of a normal-inverse-gamma distribution.Logically, this originates as follows:

The priors are normally defined as follows:

p (μ ∣ σ 2; μ 0, n 0) ∼ N (μ 0, σ 2 / n 0) p (σ 2; ν 0, σ 0 2) ∼ I χ 2 (ν 0, σ 0 2) = I G (ν 0 / 2, ν 0 σ 0 2 / 2) {\displaystyle {\begin{aligned}p(\mu \mid \sigma ^{2};\mu _{0},n_{0})&\sim {\mathcal {N}}(\mu _{0},\sigma ^{2}/n_{0})\\p(\sigma ^{2};\nu _{0},\sigma _{0}^{2})&\sim I\chi ^{2}(\nu _{0},\sigma _{0}^{2})=IG(\nu _{0}/2,\nu _{0}\sigma _{0}^{2}/2)\end{aligned}}}

The update equations can be derived, and look as follows:

x ¯ = 1 n ∑ i = 1 n x i μ 0 ′ = n 0 μ 0 + n x ¯ n 0 + n n 0 ′ = n 0 + n ν 0 ′ = ν 0 + n ν 0 ′ σ 0 2 ′ = ν 0 σ 0 2 + ∑ i = 1 n (x i − x ¯) 2 + n 0 n n 0 + n (μ 0 − x ¯) 2 {\displaystyle {\begin{aligned}{\bar {x}}&={\frac {1}{n}}\sum _{i=1}^{n}x_{i}\\\mu _{0}'&={\frac {n_{0}\mu _{0}+n{\bar {x}}}{n_{0}+n}}\\n_{0}'&=n_{0}+n\\\nu _{0}'&=\nu _{0}+n\\\nu _{0}'{\sigma _{0}^{2}}'&=\nu _{0}\sigma _{0}^{2}+\sum _{i=1}^{n}(x_{i}-{\bar {x}})^{2}+{\frac {n_{0}n}{n_{0}+n}}(\mu _{0}-{\bar {x}})^{2}\end{aligned}}}

The respective numbers of pseudo-observations add the number of actual observations to them. The new mean hyperparameter is once again a weighted average, this time weighted by the relative numbers of observations. Finally, the update for ν 0 ′ σ 0 2 ′ {\textstyle \nu _{0}'{\sigma _{0}^{2}}'} is similar to the case with known mean, but in this case the sum of squared deviations is taken with respect to the observed data mean rather than the true mean, and as a result a new interaction term needs to be added to take care of the additional error source stemming from the deviation between prior and data mean.

The prior distributions are p (μ ∣ σ 2; μ 0, n 0) ∼ N (μ 0, σ 2 / n 0) = 1 2 π σ 2 n 0 exp (− n 0 2 σ 2 (μ − μ 0) 2) ∝ (σ 2) − 1 / 2 exp (− n 0 2 σ 2 (μ − μ 0) 2) p (σ 2; ν 0, σ 0 2) ∼ I χ 2 (ν 0, σ 0 2) = I G (ν 0 / 2, ν 0 σ 0 2 / 2) = (σ 0 2 ν 0 / 2) ν 0 / 2 Γ (ν 0 / 2) exp (σ 2) 1 + ν 0 / 2 ∝ (σ 2) − (1 + ν 0 / 2) exp. {\displaystyle {\begin{aligned}p(\mu \mid \sigma ^{2};\mu _{0},n_{0})&\sim {\mathcal {N}}(\mu _{0},\sigma ^{2}/n_{0})={\frac {1}{\sqrt {2\pi {\frac {\sigma ^{2}}{n_{0}}}}}}\exp \left(-{\frac {n_{0}}{2\sigma ^{2}}}(\mu -\mu _{0})^{2}\right)\\&\propto (\sigma ^{2})^{-1/2}\exp \left(-{\frac {n_{0}}{2\sigma ^{2}}}(\mu -\mu _{0})^{2}\right)\\p(\sigma ^{2};\nu _{0},\sigma _{0}^{2})&\sim I\chi ^{2}(\nu _{0},\sigma _{0}^{2})=IG(\nu _{0}/2,\nu _{0}\sigma _{0}^{2}/2)\\&={\frac {(\sigma _{0}^{2}\nu _{0}/2)^{\nu _{0}/2}}{\Gamma (\nu _{0}/2)}}~{\frac {\exp \left}{(\sigma ^{2})^{1+\nu _{0}/2}}}\\&\propto {(\sigma ^{2})^{-(1+\nu _{0}/2)}}\exp \left.\end{aligned}}}

Therefore, the joint prior is

p (μ, σ 2; μ 0, n 0, ν 0, σ 0 2) = p (μ ∣ σ 2; μ 0, n 0) p (σ 2; ν 0, σ 0 2) ∝ (σ 2) − (ν 0 + 3) / 2 exp. {\displaystyle {\begin{aligned}p(\mu,\sigma ^{2};\mu _{0},n_{0},\nu _{0},\sigma _{0}^{2})&=p(\mu \mid \sigma ^{2};\mu _{0},n_{0})\,p(\sigma ^{2};\nu _{0},\sigma _{0}^{2})\\&\propto (\sigma ^{2})^{-(\nu _{0}+3)/2}\exp \left.\end{aligned}}}

The likelihood function from the section above with known variance is:

p (X ∣ μ, σ 2) = (1 2 π σ 2) n / 2 exp {\displaystyle {\begin{aligned}p(\mathbf {X} \mid \mu,\sigma ^{2})&=\left({\frac {1}{2\pi \sigma ^{2}}}\right)^{n/2}\exp \left\end{aligned}}}

Writing it in terms of variance rather than precision, we get: p (X ∣ μ, σ 2) = (1 2 π σ 2) n / 2 exp ∝ σ 2 − n / 2 exp {\displaystyle {\begin{aligned}p(\mathbf {X} \mid \mu,\sigma ^{2})&=\left({\frac {1}{2\pi \sigma ^{2}}}\right)^{n/2}\exp \left\\&\propto {\sigma ^{2}}^{-n/2}\exp \left\end{aligned}}} where S = ∑ i = 1 n (x i − x ¯) 2. {\textstyle S=\sum _{i=1}^{n}(x_{i}-{\bar {x}})^{2}.}

Therefore, the posterior is (dropping the hyperparameters as conditioning factors): p (μ, σ 2 ∣ X) ∝ p (μ, σ 2) p (X ∣ μ, σ 2) ∝ (σ 2) − (ν 0 + 3) / 2 exp σ 2 − n / 2 exp = (σ 2) − (ν 0 + n + 3) / 2 exp = (σ 2) − (ν 0 + n + 3) / 2 exp ∝ (σ 2) − 1 / 2 exp × (σ 2) − (ν 0 / 2 + n / 2 + 1) exp = N μ ∣ σ 2 (n 0 μ 0 + n x ¯ n 0 + n, σ 2 n 0 + n) ⋅ I G σ 2 (1 2 (ν 0 + n), 1 2 (ν 0 σ 0 2 + S + n 0 n n 0 + n (μ 0 − x ¯) 2)). {\displaystyle {\begin{aligned}p(\mu,\sigma ^{2}\mid \mathbf {X})&\propto p(\mu,\sigma ^{2})\,p(\mathbf {X} \mid \mu,\sigma ^{2})\\&\propto (\sigma ^{2})^{-(\nu _{0}+3)/2}\exp \left{\sigma ^{2}}^{-n/2}\exp \left\\&=(\sigma ^{2})^{-(\nu _{0}+n+3)/2}\exp \left\\&=(\sigma ^{2})^{-(\nu _{0}+n+3)/2}\exp \left\\&\propto (\sigma ^{2})^{-1/2}\exp \left\\&\quad \times (\sigma ^{2})^{-(\nu _{0}/2+n/2+1)}\exp \left\\&={\mathcal {N}}_{\mu \mid \sigma ^{2}}\left({\frac {n_{0}\mu _{0}+n{\bar {x}}}{n_{0}+n}},{\frac {\sigma ^{2}}{n_{0}+n}}\right)\cdot {\rm {IG}}_{\sigma ^{2}}\left({\frac {1}{2}}(\nu _{0}+n),{\frac {1}{2}}\left(\nu _{0}\sigma _{0}^{2}+S+{\frac {n_{0}n}{n_{0}+n}}(\mu _{0}-{\bar {x}})^{2}\right)\right).\end{aligned}}}

In other words, the posterior distribution has the form of a product of a normal distribution over p (μ | σ 2) {\textstyle p(\mu |\sigma ^{2})} times an inverse gamma distribution over p (σ 2) {\textstyle p(\sigma ^{2})}, with parameters that are the same as the update equations above.