  In realistic disordered systems, such as the Edwards-Anderson (EA) spin
glass, no order parameter, such as the Parisi overlap distribution, can be both
translation-invariant and non-self-averaging. The standard mean-field picture
of the EA spin glass phase can therefore not be valid in any dimension and at
any temperature. Further analysis shows that, in general, when systems have
many competing (pure) thermodynamic states, a single state which is a mixture
of many of them (as in the standard mean-field picture) contains insufficient
information to reveal the full thermodynamic structure. We propose a different
approach, in which an appropriate thermodynamic description of such a system is
instead based on a metastate, which is an ensemble of (possibly mixed)
thermodynamic states. This approach, modelled on chaotic dynamical systems, is
needed when chaotic size dependence (of finite volume correlations) is present.
Here replicas arise in a natural way, when a metastate is specified by its
(meta)correlations. The metastate approach explains, connects, and unifies such
concepts as replica symmetry breaking, chaotic size dependence and replica
non-independence. Furthermore, it replaces the older idea of non-self-averaging
as dependence on the bulk couplings with the concept of dependence on the state
within the metastate at fixed coupling realization. We use these ideas to
classify possible metastates for the EA model, and discuss two scenarios
introduced by us earlier --- a nonstandard mean-field picture and a picture
intermediate between that and the usual scaling/droplet picture.
