  It is shown that odd harmonic azimuthal correlations, including the directed
flow $v_1$, in forward two-particle production in peripheral proton-nucleus
($pA$) collisions can arise simply from the radial nuclear profile of a large
nucleus. This requires consideration of the C-odd part of the gluonic
generalized transverse momentum dependent (GTMD) correlator of nucleons in the
nucleus. The gluonic GTMD correlator is the Fourier transform of an off-forward
hadronic matrix element containing gluonic field strength tensors that are
connected by gauge links. It is parametrized in terms of various gluon GTMD
distribution functions (GTMDs). We show (in a gauge invariant way) that for the
relevant dipole-type gauge link structure in the small-$x$ limit the GTMD
correlator reduces to a generalized Wilson loop correlator. The Wilson loop
correlator is parametrized in terms of a single function, implying that in the
region of small $x$ there is only one independent dipole-type GTMD, which can
have a C-odd part. We show that the odderon Wigner distribution, which is
related to this C-odd dipole GTMD by a Fourier transform, generates odd
harmonics in the two-particle azimuthal correlations in peripheral $pA$
collisions. We calculate the first odd harmonic $v_1$ for forward production
within the color glass condensate framework in the limit of a large number of
colors. We find that nonzero odd harmonics are present without breaking the
rotational symmetry of the nucleus, arising just from its inhomogeneity in the
radial direction. Using a CGC model with a cubic action, we illustrate that
percent level $v_1$ can arise from this C-odd mechanism. In contrast, we show
that only even harmonics arise in diffractive dijet production in
ultra-peripheral $pA$ collisions where this gluon dipole GTMD also appears.
