  The so-called "replica method" of statistical physics is employed for the
large system analysis of vector precoding for the Gaussian multiple-input
multiple-output (MIMO) broadcast channel. The transmitter is assumed to
comprise a linear front-end combined with nonlinear precoding, that minimizes
the front-end imposed transmit energy penalty. Focusing on discrete complex
input alphabets, the energy penalty is minimized by relaxing the input alphabet
to a larger alphabet set prior to precoding. For the common discrete
lattice-based relaxation, the problem is found to violate the assumption of
replica symmetry and a replica symmetry breaking ansatz is taken. The limiting
empirical distribution of the precoder's output, as well as the limiting energy
penalty, are derived for one-step replica symmetry breaking. For convex
relaxations, replica symmetry is found to hold and corresponding results are
obtained for comparison. Particularizing to a "zero-forcing" (ZF) linear
front-end, and non-cooperative users, a decoupling result is derived according
to which the channel observed by each of the individual receivers can be
effectively characterized by the Markov chain u-x-y, where u, x, and y are the
channel input, the equivalent precoder output, and the channel output,
respectively. For discrete lattice-based alphabet relaxation, the impact of
replica symmetry breaking is demonstrated for the energy penalty at the
transmitter. An analysis of spectral efficiency is provided to compare discrete
lattice-based relaxations against convex relaxations, as well as linear ZF and
Tomlinson-Harashima precoding (THP). Focusing on quaternary phase shift-keying
(QPSK), significant performance gains of both lattice and convex relaxations
are revealed compared to linear ZF precoding, for medium to high
signal-to-noise ratios (SNRs). THP is shown to be outperformed as well.
