  Understanding the glassy nature of neural networks is pivotal both for
theoretical and computational advances in Machine Learning and Theoretical
Artificial Intelligence. Keeping the focus on dense associative Hebbian neural
networks, the purpose of this paper is two-fold: at first we develop rigorous
mathematical approaches to address properly a statistical mechanical picture of
the phenomenon of {\em replica symmetry breaking} (RSB) in these networks, then
-- deepening results stemmed via these routes -- we aim to inspect the {\em
glassiness} that they hide. In particular, regarding the methodology, we
provide two techniques: the former is an adaptation of the transport PDE to the
case, while the latter is an extension of Guerra's interpolation breakthrough.
Beyond coherence among the results, either in replica symmetric and in the
one-step replica symmetry breaking level of description, we prove the Gardner's
picture and we identify the maximal storage capacity by a ground-state analysis
in the Baldi-Venkatesh high-storage regime.
  In the second part of the paper we investigate the glassy structure of these
networks: in contrast with the replica symmetric scenario (RS), RSB actually
stabilizes the spin-glass phase. We report huge differences w.r.t. the standard
pairwise Hopfield limit: in particular, it is known that it is possible to
express the free energy of the Hopfield neural network as a linear combination
of the free energies of an hard spin glass (i.e. the Sherrington-Kirkpatrick
model) and a soft spin glass (the Gaussian or "spherical" model). This is no
longer true when interactions are more than pairwise (whatever the level of
description, RS or RSB): for dense networks solely the free energy of the hard
spin glass survives, proving a huge diversity in the underlying glassiness of
associative neural networks.
