  Modern statistical modeling is an important complement to the more
traditional approach of physics where Complex Systems are studied by means of
extremely simple idealized models. The Minimum Description Length (MDL) is a
principled approach to statistical modeling combining Occam's razor with
Information Theory for the selection of models providing the most concise
descriptions. In this work, we introduce the Boltzmannian MDL (BMDL), a
formalization of the principle of MDL with a parametric complexity conveniently
formulated as the free-energy of an artificial thermodynamic system. In this
way, we leverage on the rich theoretical and technical background of
statistical mechanics, to show the crucial importance that phase transitions
and other thermodynamic concepts have on the problem of statistical modeling
from an information theoretic point of view. For example, we provide
information theoretic justifications of why a high-temperature series expansion
can be used to compute systematic approximations of the BMDL when the formalism
is used to model data, and why statistically significant model selections can
be identified with ordered phases when the BMDL is used to model models. To
test the introduced formalism, we compute approximations of BMDL for the
problem of community detection in complex networks, where we obtain a
principled MDL derivation of the Girvan-Newman (GN) modularity and the
Zhang-Moore (ZM) community detection method. Here, by means of analytical
estimations and numerical experiments on synthetic and empirical networks, we
find that BMDL-based correction terms of the GN modularity improve the quality
of the detected communities and we also find an information theoretic
justification of why the ZM criterion for estimation of the number of network
communities is better than alternative approaches such as the bare minimization
of a free energy.
