\section{Introduction}
Neuro-imaging techniques have allowed the reconstruction of structural
human brain networks, composed of hundreds of neural regions and
thousands of white-matter fiber interconnections.  The resulting
``human connectome'' (HC) \cite{Hagmann,Honey09} turns out to be
organized in moduli --characterized by a much larger intra than inter
connectivity-- structured in a hierarchical nested fashion across many
scales
\cite{Bullmore-Sporns,Sporns,Review-Kaiser,Review-Bullmore,Buzsaki,Zhou06,
Ivkovic,Sporns2014}.
On the other hand, ``functional'' connections between nodes in these
networks have been empirically inferred from correlations in neural
activity as detected in electroencephalogram and functional magnetic
resonance time series. Unveiling how structural and functional
networks influence and constrain each other is a task of outmost
importance. A few pioneering works found that the hierarchical-modular
organization of structural brain networks has profound implications
for neural dynamics
\cite{Zhou06,Zhou07,Kaiser07,Kaiser10,Rubinov}. For example, neural
activity propagates in hierarchical networks in a rather distinctive
way, not observed on simpler networks \cite{Nat-Comm}; beside the
usual two phases --percolating and non-percolating-- commonly
encountered in models of activity propagation, an intermediate
``Griffiths phase'' \cite{Vojta-Review} emerges on the hierarchical HC
network \cite{GPCN,Nat-Comm,GPCN2}.  Such a Griffiths phase stems from the
existence of highly diverse relatively-isolated moduli or ``rare
regions'' where neural activity remains mostly localized generating slow
dynamics and very large responses to perturbations \cite{Vojta-Review,Nat-Comm,GPCN,GPCN2}.

Brain function requires coordinated or coherent neural activity at a
wide range of scales, thus, neural synchronization is a major theme in
neuroscience \cite{Buzsaki,Bennett2004,Breakspear-multiscale}.
Synchronization plays a key role in vision \cite{Sompolinsky88},
memory \cite{Klimesch1996}, neural communication
\cite{Deco-inf-transf}, and other cognitive functions
\cite{Niebur2000}. An excess of synchrony results in pathologies such
as epilepsy or Parkinsonian disease, while neurological deficit of
synchronization has been related to autism and schizophrenia
\cite{Kandel00}.

Our aim here is to scrutinize the special features of synchronization
dynamics \cite{RPK-book} --as exemplified by the canonical Kuramoto
model \cite{Kuramoto75,Strogatz00,Acebron-Review}-- running on top of
the best available human connectome mapping
\cite{Hagmann,Honey09,Arenas-Review}. This consists of a network of
$998$ nodes, each of them representing a mesoscopic population of
neurons --able to produce self-sustained oscillations
\cite{Sporns2011}-- whose mutual connections are encoded by a
symmetric weighted connectivity matrix $\bf{W}$
\cite{Hagmann,Honey09}.  The validity of this admittedly simplistic
Kuramoto model as a convenient tool to explore the generic features of
complex brain dynamics at a large scales has been recently emphasized
in the literature \cite{Breakspear2010,Sporns2011,Arenas10cat}.  Here,
we uncover the existence of a novel intermediate phase for
synchronization dynamics --similar in spirit to the Griffiths phases
discussed above-- which stems from the hierarchical modular
organization of the HC and which gives rise to very complex and rich
synchronization dynamical patterns.  We identify this novel phase as
the optimal regime for the brain to harbor complex behavior, large
dynamical repertoires, and optimal trade-offs between local
segregation and global integration.

The Kuramoto dynamics on a generic network (see \cite{Arenas-Review}
for a nice and comprehensive review) is defined by:
\begin{equation}
\dot{\theta}_{i}(t)=\omega_{i}+ k \sum_{j=1}^{N}
W_{ij}\sin\left[\theta_{j}(t)-\theta_{i}(t)\right], \label{eq:Kuramoto}
\end{equation}
where ${\theta}_{i}(t)$ is the phase at node $i$ at time $t$. The
intrinsic frequencies $\omega_{i}$ --accounting for node
heterogeneity-- are extracted from some arbitrary distribution
function $g(\omega)$, $W_{ij}$ are the elements of the $N \times N$
connectivity matrix $\bf{W}$, and $k$ is the coupling strength.  Time
delays, noise, and phase frustration could also be straightforwardly
implemented.  The Kuramoto order parameter is defined as
$Z(t)=R(t)\mathrm{e}^{i\psi(t)} = \langle \mathrm{e}^{i \theta_i(t)}
\rangle$, where $0 \leq R(t) \leq 1$ gauges the overall coherence and
$\psi(t)$ is the average global phase.  In large populations of
well-connected oscillators  without frequency dispersion,
perfect coherence ($R =1$) emerges for any coupling strength; on the
other hand, frequency heterogeneity leads to a phase transition at
some critical value of $k$, separating a coherent steady state from an
incoherent one
\cite{Kuramoto75,Strogatz00,Acebron-Review,Arenas-Review}.  Analytical
insight onto this phase transition can be obtained using the
celebrated Ott-Antonsen (OA) ansatz, allowing for a projection of the
high-dimensional dynamics into an evolution equation for $Z(t)$ with
remarkable accuracy in the large-$N$ limit \cite{OA,Skardal-Restrepo}.
\begin{figure}[htbp]
\centering
\includegraphics[width=12cm,angle=0]{Figure1.pdf}
\caption{Global synchronization dynamics in the human connectome.
  a) Time average of the order parameter $R(t)$,
  for Kuramoto dynamics on the HC network for a specific and fixed set
  of frequencies extracted from a $N(0,1)$ Gaussian distribution. A
  broad intermediate regime separates the incoherent phase (low $k$)
  from the synchronous one (high $k$). In this regime, coherence
  increases with $k$ in an intermittent fashion, and with strong
  dependence on the frequency realization.  b) Raster plot of
  individual phases (vertical axis) showing local rather than global
  synchrony and illustrating the coexistence of coherent and
  incoherent nodes ($k=2.7$) as time runs.  c) $R(t)$ for $4$ values
  of $k$ (arrows in the main plot).  d) Adjacency matrix of the HC
  network with nodes ordered to emphasize its modular structure as
  highlighted by a community detection algorithm (main text), keeping
  the partition into the $2$ hemispheres (dashed lines). Intra-modular
  connections (shown in color) are dense while inter-modular ones
  (grey) are limited to tiny subsets, acting as interfaces between
  moduli.  Integration between hemispheres is mostly carried out by
  the $3$ central moduli. This plot visually illustrates the
  hierarchical modular organization of the human connectome network.}
\end{figure}
\section*{Results}
\subsection*{Novel phase between order and disorder in the HC}
We have performed a computational study of the Kuramoto model running
on top of the HC network (details are given in the Methods section).
Our results reveal the existence of an intermediate regime placed
between the coherent and the incoherent phase (see Fig.1).  This is
characterized by broad quasi-periodic temporal oscillations of $R(t)$
which wildly depend upon the realization of intrinsic frequencies
\cite{Arenas1,Arenas2}.  Anomalously large sampling times would be
required to extract good statistics for the actual mean values and
variances.  Collective oscillations of $R(t)$ are a straightforward
manifestation of partial synchronization and they are robust against
changes in the frequency distribution (e.g. Gaussian, Lorentzian,
uniform, etc.)  whereas the location and width of the intermediate
phase depend upon details.  As this phenomenology is reminiscent of
Griffiths phases --posed in between order and disorder and stemmig
from the existence of semi-isolated regions
\cite{Vojta-Review,GPCN,Nat-Comm}-- it is natural to investigate how
the HC hierarchical modular structure affects synchronization
dynamics.

Any network with perfectly isolated and independently synchronized
moduli trivially exhibits oscillations of $R(t)$, with amplitude
peaking at times when maximal mutual synchronization happens to be
incidentally achieved. Such oscillations can become chaotic if a
finite and relatively small number of different coherent moduli are
coupled together \cite{Popovych}. Thus, in a connected network without
delays or other additional ingredients, oscillations in the global
coherence are the trademark of strong modular structure with weakly
interconnected moduli.

Strong modular organization into distinct hierarchical levels is
indeed present in the HC as reveled by standard community detection
algorithms \cite{Radatools,Ivkovic} and as already discussed in the
literature (see e.g. \cite{Nat-Comm} and references therein).  For
instance, we have found that the optimal partition into disjoint
communities --i.e. the partition maximizing the modularity parameter
\cite{Newman-Review}-- corresponds to a division in $12$ communities
(see Fig.1d) while, at a higher hierarchical level, a separation into
just $2$ moduli --the $2$ cerebral hemispheres-- is obtained
\cite{Honey09} (Fig.1d).  Obviously these $2$ coarser moduli include
the $12$ above as sub-moduli.  Although more levels of hierarchical
partitioning could be inferred (see e.g. \cite{Sporns2014} and
refs. therein), for the sake of simplicity we focus on these two
levels $l$, $l=1$ and $l=2$ with $12$ and $2$ moduli, respectively.
\begin{figure}[ht]
\begin{center}
\vspace{-0cm}
   \includegraphics[width=12cm]{Figure2.pdf}
   \caption{Two-block model.
     (a) Sketch of the two-block model.  (b)
     Global order parameter for the two-block model with
     $M=128$ and two interfacial nodes.
     Results of the numerical integration of the $258$
     Kuramoto equations (blue points) are in strikingly good agreement
     with the integration of Eqs.(\ref{OA}) (solid blue line). Local
     block-wise order parameters are shown for comparison (small
     symbols; dashed lines are guides to the eye).  A first
     transition, where local order emerges, occurs at $k\approx 0.02$,
     while global coherence is reached at $k \approx 90$.  In the
     intermediate region, $R(t)$ oscillates (inset), revealing the
     lack of global coherence.  Despite the simplicity of this toy
     model, these results constitute the essential building-block upon
     which further levels of complexity rely (see main text).}
    \label{fig:blocks}
\end{center}
\end{figure}

\subsection*{A simplistic model for global oscillations}
To shed further light on the properties of synchronization on the HC,
we consider a very simple network model --allowing for analytical
understanding-- which will constitute the elementary
``building-block'' for subsequent more complex analyses.  This
consists of a few blocks with very large internal connectivity and
very sparse inter-connectivity. Each block is composed by a bulk of $M
\gg 1$ nodes that share no connection with the outside and a
relatively small ``interfacial'' set that connects with nodes in other
blocks. For instance, in the simplest realization, consisting of just
two blocks connected by a single pair of nodes (Fig.2), each block is
endowed with local coherence $r_{\mathrm{A},\mathrm{B}}$, average
phase $\psi_{\mathrm{A},\mathrm{B}}$, and average characteristic
frequency $\omega_\mathrm{A,B}$, while 1-node interfaces have perfect
coherence $r=1$, phase $\varphi_{\mathrm{A},\mathrm{B}}$, and
characteristic frequency $\nu_{\mathrm{A},\mathrm{B}}$.  In this case,
$N=2M+2$, and the OA ansatz can be safely applied to each block (large
$M$) but not to single-node interfaces.  In the particular case
(convenient for analytical treatment) in which $g(w)$ are zero-mean
Lorentz distributions $g(\omega) = \frac{1}{\pi}
\frac{\delta}{(\omega-\Omega_0)^2 + \delta^2}$ with spreads
$\delta_{\mathrm{A},\mathrm{B}}$, the resulting set of OA equations
can be easily shown to be:
 \begin{eqnarray}
  \dot{\psi}_\mathrm{A} & =&
  \omega_{\mathrm{A}} + k
  \frac{1+r_\mathrm{A}^2}{2r_\mathrm{A}}
  \sin(\varphi_\mathrm{A}-\psi_\mathrm{A}) \nonumber \\
  \dot{r}_\mathrm{A}  &=&
  -\delta_\mathrm{A}r_\mathrm{A} + k
  \frac{1-r_\mathrm{A}^2}{2}
  \left[Mr_\mathrm{A}+
  \cos(\varphi_\mathrm{A}-\psi_\mathrm{A})\right] \nonumber \\
\dot{\varphi}_\mathrm{A} & = &
  \nu_\mathrm{A}+
k \left[
Mr_\mathrm{A}\sin(\psi_\mathrm{A}-\varphi_\mathrm{A})+\sin(\varphi_\mathrm{B}
    -\varphi_\mathrm{A})\right]
\label{OA}
\end{eqnarray}
(together with $r=1$ for each 1-node interface), and a symmetric set ($A
\leftrightarrow B$) for block $\mathrm{B}$.  The solution of Eq.(2)
--displayed in Fig.2-- reveals a transition to local coherence within
each block at a certain threshold value of $k\approx 0.02$. As soon as
local order is attained, $r_\mathrm{A,B}\approx 1$ and
$\dot{\psi}_\mathrm{A,B}\approx 0$, from Eq.(2) the mutual
synchronization process obeys
 \begin{equation}
 \dot{\varphi}_\mathrm{A}\approx
 (\nu_\mathrm{A}+M\omega_{\mathrm{A}})+
 k\sin\left(\varphi_\mathrm{B}-\varphi_\mathrm{A}\right)
 \end{equation}
 and a symmetrical equation for $\dot{\varphi}_\mathrm{B}$. For small
 $k$, the right-hand side is dominated by
 $\nu_\mathrm{A}+M\omega_{\mathrm{A}}$: whereas the average value
 $\omega_\mathrm{A}$ becomes arbitrarily small within blocks (assuming
 that $M$ is large), the frequency $\nu_\mathrm{A}$ does not.
 Consequently, synchronization between the two blocks through the
 interfacial link is frustrated: each block remains internally
 synchronized but is unable to achieve coherence with the other over a
 broad interval of coupling strengths. This interval is delimited
 above by a second transition at $k \sim
 \max\{M|\omega_\mathrm{A,B}|,\nu_{\mathrm{A,B}}\}$, where $k$ is
 large enough as to overcome frustration and generate global
 coherence. This picture is confirmed by numerical integration of the
 full system of $N$ coupled Kuramoto equations as well as by its OA
 approximation (Eq.(2)), both in remarkably good agreement. Therefore,
 local and global coherences have their onsets at two well-separated
 transition points \cite{Skardal-Restrepo} and --similarly to the much
 more complex HC case-- $R$ oscillates in the intermediate regime
 (Fig.2).  Similar results hold for versions of the model with more
 than two moduli (e.g. $4$; see below).  The existence of two distinct
 (local and global) transitions had already been reported in a recent
 study of many blocks with much stronger inter-moduli connections than
 here \cite{Skardal-Restrepo} (even if, owing to this difference, no
 sign of an intermediate oscillatory phase was reported).

 In particular, the value of two-block models has already been
 explored in the past, for systems of identical oscillators with
 non-zero phase lags, in which each node is coupled equally to all the
 others in its community, and less strongly to those in the other
 \cite{Abrams2008}. In such systems, local coherence emerged for large
 enough values of the phase lag.

 Our two-block model shows that the presence of ``structural
 bottlenecks'' between moduli combined with heterogeneous frequencies
 at their contact nodes (interfaces) are essential ingredients to
 generate a broad region of global oscillations in $R$, even in the
 absence of phase lag.  Still, it is obviously a too-simplistic model
 to account for all the rich phenomenology emerging on the HC, as we
 show now.
\begin{figure}[htbp]
\centering
\includegraphics[width=12cm,angle=0]{Figure3.pdf}
\caption{Local synchronization in the human connectome.
  a) Oscillations of the local order parameters
  (``chimera-like states'') in one particular modulus in the partitions of
  the HC into $12$ (green, $l=1$, and $k=3$) and $2$ (magenta, $l=2$,
  and $k=10$) moduli, respectively. The characteristic frequency of
  these oscillations is typically between $0.01$ and $0.1 Hz$ (a range
  which coincides with slow modes detected in brain activity; see
  e.g. \cite{Breakspear2010}).  b) Average of the local order
  parameter over all moduli and c) chimera index for moduli at levels
  as in a), as a function of $k$.  Global order (thin black line in
  b)) emerges only after local order is attained at lower levels. d)
  Average decay of activity $\rho$ for identical frequencies
  $\omega=0$ in the HC network and comparison with a single-level
  modular network (made up of $4$ similar random moduli at a single
  hierarchical level) of the same size and average connectivity as the
  HC network. Symbols stand for different values of $k$.  e)
  Characteristic decay times corresponding to the inverse of the first
  $1000$ non-trivial eigenvalues of the Laplacian matrix (x axis) as a
  function of their respective ordered indices (y axis), for networks
  as in d). The stretched exponential behavior in d) is the result of
  the convolution of slow time scales associated with small
  eigenvalues in e).}
\end{figure}
\subsection*{Oscillations of local coherence in the HC}
Fig.3 shows numerical results for the local order parameter $r^{(l)}$
for some of the moduli at the $2$ hierarchical levels, $l=1$ and $l=2$
in the HC network.  It reveals that (Fig.3a) local coherences exhibit
oscillatory patterns in time (with characteristic frequencies
typically between $0.01$ and $0.1 Hz$) and that (Fig.3b) the
transition to local coherence at progressively higher hierarchical
level occurs at progressively larger values of $k$; i.e. coherence
emerges out of a hierarchical bottom-up process as illustrated above
for the for the two-block model (see
\cite{Arenas_scales,Skardal-Restrepo}).  Observe, however, that local
oscillations were not present in the two-block model.  This suggests
that the $12$ moduli in the HC are on their turn composed of finer
sub-moduli and that structural frustration, as introduced above,
affects all hierarchical levels.  The average variance of local
coherences (called chimera index, $\chi$) \cite{Shanahan2010} exhibits
a marked peak --reflecting maximal configurational variability-- at
the transition point for the corresponding level (Fig. 3b-c and
Methods section).  Similar intra-modular oscillatory patterns --dubbed
{\it chimera states}-- have been recently found
\cite{Abrams2008,Abrams2004,Shanahan2010,Shanahan2012} in Kuramoto
models in which explicit phase lags induce a different kind of
frustration, hindering global synchronization. Strictly speaking, chimeras
are defined in systems of identical oscillators. In such a case, a non-zero
phase lag term is essential for partial synchronization to occur.
Realistic models of the brain, however, require oscillators to be
heterogenous. States of partial synchronization in empirical brain networks
with frequency heterogeneity have been found for Kuramoto models with explicit
time delays \cite{Sporns2011}. In contrast, the
chimera-like states put forward here have a purely structural origin, as they
arise from the network topology. It was noted in the past that synchronization
in a synthetic network with hubs could be limited to those hubs by tuning
clustering properties, and global order could be attained in a monotonous
step-like fashion upon increasing $k$ \cite{McGraw2005}.
Fig.3b instead reveals that the ordering process in the hierarchical modular HC
may be
non-monotonous: coherence does not systematically grow with $k$. Indeed, the
emergence of local order in some community may hinder or reduce coherence in
others, inducing local ``desynchronization'' and reflecting the
metastable nature of the explored states.

\subsection*{Anomalous dynamics on the HC} Fixing all intrinsic
frequencies to be identical allows us to focus specifically on
structural effects. Thus, we consider, without loss of generality, the
simple case $\omega_i=0$, and define the ``activity'' $\rho=1-\langle
R\rangle$. In this case, perfect asymptotic coherence should emerge
for all values of $k$ but, as illustrated in Fig. 3d, the convergence
towards $\rho=0$ turns out to be extremely slow (much slower than
exponential). This effect can be analytically investigated assuming
that, for large enough times, all phase differences are relatively
small. Then, up to first order, $\dot{\theta}_i = - k \sum_j
L_{ij}\theta_j$ where $L_{ij}=\delta_{ij}\sum_l W_{jl}-W_{ij}$ are the
elements of the Laplacian matrix \cite{FanChung,JStat2006}. Solving the linear
problem,
$\theta_i(t)= \sum_{l,j}\mathrm{e}^{- k \lambda_l
  t}v^l_iv^l_j\theta_j(0)$, where $\lambda_l$ denotes the $l$-th
Laplacian eigenvalue ($0=\lambda_1 < \lambda_2 < ... < \lambda_N$) and
$v^l_i$ the $i$-th component of the corresponding eigenvector. Given
that the averaged order parameter can be written as $Z (t) \approx \sum_j
(1+\mathrm{i}\theta_j-\frac{1}{2}\theta_j^2)/N$, averaging over
initial conditions, and considering that (as the Laplacian has zero
row-sums \cite{JStat2006}) $\lambda_1=0$, we obtain
\begin{equation}\label{eq:discrete_lambda}
\rho(t) =
\frac{\sigma^2}{2}\sum_{l=2}^N \mathrm{e}^{-2 k \lambda_l t},
\end{equation}
where $\sigma$ is the standard deviation of the initial phases. This
expression holds for any connected network.  As usual, the larger the
spectral gap $\lambda_2$, the more ``entangled'' \cite{JStat2006} the
network and thus the more difficult to divide it into well separated
moduli ($\lambda_2=0$ only for disconnected networks)
\cite{JStat2006,FanChung}.  For large spectral gaps all timescales are
fast, and the last expression can be approximated by its leading
contribution, ensuing exponential relaxation to $\rho=0$, as in fact
observed in well-connected network architectures (Erd\H os-R\'enyi,
scale free, etc. \cite{Newman-Review}). This is not the case for the
HC matrix, for which a tail of small non-degenerate eigenvalues is
encountered (see Fig.3e and \cite{Nat-Comm}).  Each eigenvalue
$\lambda_i$ in the tail corresponds to a natural division of moduli
into sub-moduli \cite{JStat2006}, and the broad tail reflects the
heterogeneity in the resulting modular sizes. As a consequence, each
of these eigenvalues --with its associated large timescale, $t_i =
1/\lambda_i$-- contributes to the sum above, giving rise to a
convolution of relaxation processes, entailing anomalously-slow
dynamics, which could not be explained by a single-level modular
network (see Fig.3d-e): slow dynamics necessarily stems from the
existence of a hierarchy of moduli and structural bottlenecks.  As
explained in Methods, in the case of the HC the convolution of
different times scales gives rise  to stretched-exponential decay,
which perfectly fits with numerical results in Fig.3.
It was noted in the past that strongly modular networks exhibit
isolated eigenvalues in the lower edge of the laplacian
spectrum. Synchronization would develop in a step-wise process in
time, where each transient would be given by each isolated eigenvalue
\cite{Arenas_scales}.  In our case, the depth of the hierarchical
organization and the strength of topological disorder produce instead
a quasi-continuous tail of eigenvalues, and the step-wise process is
replaced by an anomalous stretched-exponential behavior.

\subsection*{A more refined model: hierarchical modular synthetic
  networks} To shed additional light on the previous findings for the
HC --i.e. the emergence of chimera-like states and anomalously slow
dynamics-- we suggest to go beyond the single-level modular network
model and study hierarchical modular networks (HMN) in which moduli
exists within moduli in a nested way at various scales
\cite{Bullmore-Sporns,Sporns,Review-Kaiser,Review-Bullmore,Zhou06,Ivkovic}.
HMN are assembled in a bottom-up fashion: local fully-connected moduli
(e.g. of $16$ nodes) are used as building blocks.  They are
recursively grouped by establishing additional inter-moduli links in a
level-dependent way as sketched in Fig.4(top) \cite{Nat-Comm,Zhou11}.
\begin{figure}[htbp]
  \centering
\includegraphics[width=12cm,angle=0]{Figure4.pdf}
\caption{Synchronization in hierarchical modular networks.
  Top panel: sketch of the HMN
  model. At hierarchical level $1$, $2^s$ basal fully connected blocks
  of size $M$ are linked pairwise into super-blocks by establishing a
  fixed number $\alpha$ of random unweighted links between the
  elements of each ($\alpha=2$ in the Fig.). Newly formed blocks are
  then linked iteratively with the same $\alpha$ up to level $s$,
  until the network becomes connected.  a), b), c) as in Fig.3, but
  for a HMN with $N=512$, $s=5$, and $\alpha=4$.  Hierarchical levels
  are $i=1\to 5$ in black, blue, green, magenta and red respectively
  (not all shown in a) for clarity).  d) Time relaxation of activity
  $\rho$ for homogeneous characteristic frequencies $\omega=0$, for
  logarithmically equally spaced values of $k$.  Averages over $10^6$
  realizations of HMNs with $N=4096$ and $s=11$.  Inset: as in the
  main plot d), but representing as a function of $t^{1/2}$ and
  confirming the predicted stretched exponential behavior. e) Inverse
  tail-eigenvalues (as in Fig.3) for a HMN as in e).
}
\end{figure}
Our computational analyses of the Kuramoto dynamics on HMN substrates
(see Fig.4) reveal: (i) a sequence of synchronization transitions for
progressively higher hierarchical levels at increasing values of $k$,
(ii) chimera-like states at every hierarchical level, resulting in a
hierarchy of metastable states with maximal variability at the
corresponding transition points, (iii) extremely slow relaxation
toward the coherent state when all internal frequencies are
identical. Furthermore, anomalies in the Laplacian spectrum analogous
to those of the HC network are observed for HMN matrices; in
particular, the lower edge of the HMN Laplacian spectrum has been
recently shown to exhibit a continuous exponential Lifshitz tail
$p(\lambda)\sim \mathrm{e}^{-1/\lambda^a}$ for $N \rightarrow \infty$,
with $a\approx 1$ \cite{Nat-Comm}.  Taking the continuum limit of
Eq.(\ref{eq:discrete_lambda}), we find $\rho(t)\approx
\frac{\sigma^2}{2}\int d\lambda\, p(\lambda) {e}^{-2 k\lambda t}$,
which can be evaluated with the saddle-point method (see Methods),
leading to
\begin{equation}\label{eq:stretched}
\rho(t) \sim \mathrm{e}^{- \sqrt{8 k t}},
\end{equation}
i.e. anomalous stretched-exponential asymptotic behavior, in excellent
agreement with computational results (see Fig.4d).  Therefore,
hierarchical modular networks constitute a parsimonious and adequate
model for reproducing all the complex synchronization phenomenology of
the HC.

A crucial role in the emergence of such behavior is played by disorder.
One would be tempted to believe that all networks characterized by a finite
spectral dimension could potentially give rise to this phenomenology. This is
obviously not the case for a regular lattice, where the spectral gap is always
well defined. A fractal lattice or an ordered tree, on the other hand, could
exhibit a hierarchy of discrete low eigenvalues, whose multiplicities reflect
system symmetries. The introduction of disorder, as in HMNs, is then necessary
in order to transform such hierarchy of discrete levels into a continuous
Lifshitz tail, leading eventually to the behavior predicted by Eq.
(\ref{eq:stretched}).

\section*{Discussion}

Simple models of synchronization dynamics exhibit an unexpectedly rich
phenomenology when operating on top of empirical human brain networks.
This complexity includes oscillatory behavior of the order parameter
suggesting the existence of relatively isolated structural communities
or moduli, that --as a matter of fact-- can be identified by using
standard community detection algorithms. Even more remarkably,
oscillations in the level of internal coherence are also present
within these moduli, suggesting the existence of a whole hierarchy of
nested levels of organization, as also found in the recent literature
relying on a variety of approaches
\cite{Bullmore-Sporns,Sporns,Review-Kaiser,Review-Bullmore,Buzsaki,Zhou06,
Ivkovic,Sporns2014}.
Aimed at unveiling this complex behavior we have introduced a family
of hierarchical modular networks and studied them in order to assess
what structural properties are required in order to reproduce the complex
synchronization patterns observed in brain networks.

In the absence of frequency dispersion, perfect coherence is achieved
in synthetic hierarchical networks by following a bottom-up ordering
dynamics in which progressively larger communities --with inherently
different timescales-- become coherent (see
\cite{Arenas_scales}). However, this hierarchically nested
synchronization process is constrained and altered by structural
bottlenecks --as carefully described here for the simpler two-block
toy model-- at all hierarchical levels.  This structural complexity
brings about anomalously-slow dynamics at very large timescales.
Observe that the HC, in spite of being a coarse-grained mapping of a
brain network, already shows strong signals of this ideal hierarchical
architecture as reflected in its anomalously slow synchronization
dynamics as well as in the presence of non-degenerate eigenvalues in
the lower edge of its Laplacian spectrum, acting as a fingerprint of
structural heterogeneity and complexity.  We stress that such a
complex phenomenology would be impossible to obtain in networks with
stronger connectivity patterns (e.g. with the small world property)
such as scale free-networks or high-degree random graphs. Even the
generic presence of simple communities may not be sufficient to grant
the emergence of frustration: the uniqueness of the human connectome,
and of hierarchical modular networks in general, resides in the strong
separation into distinct levels, which the synchronization dynamics is
able to resolve only at well-separated values of the coupling $k$.

On the other hand, in the presence of intrinsic frequency
heterogeneity, the described slow ordering process is further
frustrated. Actually, for small values of the coupling constant $k$
the system remains trapped into metastable and chimera-like states with
traits of local coherence at different hierarchical levels. In this
case, inter-moduli frequency barriers need to be overcome before
weakly connected moduli achieve mutual coherence. This is clearly
exemplified by the separation between distinct peaks in the chimera
index $\chi^{(l)}$ in Figs. 3-4, each one signaling the onset of an
independent synchronization process at a given level (see
Methods). The result is a complex synchronization landscape, which is
especially rich and diverse in the intermediate regime put forward
here.

Including other realistic ingredients such as explicit phase
frustration \cite{Shanahan2010} or time delays
\cite{Sporns2011,Shanahan2012} to our simplistic approach should only
add complexity to the structural frustration effect reported here. It
is also expected that more refined models --including neuro-realistic
ingredients leading to collective oscillations-- would generate
similar results, but this remains to be explored in future works.

Addition of noise to the Kuramoto dynamics would allow the system to
escape from metastable states. Stochasticity can overcome the
``potential barriers'' between mutually incoherent moduli as well as
re-introduce de-synchronization effects.  These combined effects can
make the system able to explore the nested hierarchy of attractors,
allowing one to shed some light into the complex synchronization
patterns in real brain networks. Actually, spontaneous dynamical
fluctuations have been measured in the resting state of human brains
\cite{RS}; these are correlated across diverse segregated moduli and
characterized by very slow fluctuations, of typical frequency $<0.1Hz$,
in close agreement with those found here (Fig.3).  Accordingly, it has
been suggested that the brain is routinely exploring different states
or attractors \cite{Deco2012} and that --in order to enhance
spontaneous switching between attractors-- brain networks should
operate close to a critical point, allowing for large intrinsic
fluctuations which on their turn entail attractor ``surfing'' and give
access to highly varied functional configurations
\cite{Chialvo10,Shew09,Deco2012,Haimovici,Shriki2013} and, in
particular, to maximal variability of phase synchrony \cite{Yang2012}.

The existence of multiple attractors and noise-induced surfing is
largely facilitated in the broad intermediate regime first elucidated
here, implying that a precise fine tuning to a critical point
might not be required to guarantee functional avantages usually
associated with criticality \cite{Chialvo10,Beggs08,Plenz2013}: the
role usually played by a critical point is assumed by a broad
intermediate region in hierarchically architectured complex systems
\cite{Nat-Comm}.  Finally, let us remark that our results might also
be of relevance for other hierarchically organized systems such as
gene regulatory networks \cite{Escherichia} for which coherent
activations play a pivotal role \cite{Kauffman08}.

\section*{Methods}
\subsection*{Numerical simulation of the Kuramoto model}
The Kuramoto model is simulated by numerically integrating
Eq.~(\ref{eq:Kuramoto}). Computations are carried out using both a 4th
order Runge-Kutta method of fixed step size $h=10^{-3}$ and an 8th
order Dormand-Prince method with adaptive step size. Both methods lead
to compatible results within precision limits. The robustness of the
observation of a novel intermediate phase --between incoherent and
coherent ones-- is assessed by choosing different functional forms for
the frequency distribution $g(\omega)$ (Lorentzian, Gaussian,
uniform), and by implementing variations of Eq.~\ref{eq:Kuramoto} in
which the matrix ${\bf W}$ is weight-normalized \cite{Arenas-Review}
for simulations of the HC and degree-normalized for simulations in
HMNs. No qualitative change in the phenomenology is observed.

\subsection*{Chimera index $\chi^{(l)}$ and hierarchical synchronization}
In the main text, the chimera index $\chi^{(l)}$ is introduced as a measure of
partial
synchronization at the community level $l$. At any hierarchical level $l$, a
hierarchical network
can be divided into a set of communities. Following \cite{Shanahan2010},
$\chi^{(l)}$ is
defined as follows: (i) in the steady (oscillatory) state, and for each time
$t$, local order
parameters $r_i^{(l)}(t)$ for each community $i$ are calculated and their
variance across
communities $\sigma_{\mathrm{chi}}^{(l)}(t)$
is stored; (ii) the chimera index is computed as the time average
$\chi^{(l)}=\langle \sigma_\mathrm{chi}^{(l)}(t) \rangle_t$. Having
$\chi^{(l)}>0$ at a given
hierarchical level $l$  implies that local
order is only partial as $r_i^{(l)}$ fluctuates, giving rise to a chimera-like
state. On the other hand,
$\chi^{(l)}=0$  means that each local order parameter at that level is
$r_i^{(l)}\approx 1$,
and local order has been attained. Figs. 3b-c and 4b-c show that at each $l$
(each color) a peak
in the corresponding $\chi^{(l)}$ marks the onset of the local synchronization
processes:
as soon as the peak vanishes upon increasing $k$, local order at that level is
attained.
The sequence of separated peaks in $\chi^{(l)}$ for increasing values of $l$ is
the direct evidence
of a hierarchical synchronization process.

\subsection*{Lifshitz tail and stretched-exponential asymptotic behavior}
In sparse HMNs, the lower end of the Laplacian spectrum is
characterized by an exponential tail in the density of states
$p(\lambda)\sim \mathrm{e}^{-1/\lambda^a}$, known as Lifshitz tail
\cite{Nat-Comm}.  In graphs, Lifshitz tails signal the existence of
non-trivial heterogeneous localized states governing the asymptotic
synchronization dynamics at very large times $t$. In the main text we
have shown that in the absence of frequency heterogeneity, the
$t\to\infty$ behavior of the activity is given by $\rho(t)\approx
\frac{\sigma^2}{2}\int d\lambda\, p(\lambda) {e}^{-2 k \lambda t}$.
This expression can be evaluated by applying the saddle point method,
yielding
\begin{equation}
\rho(t)\approx\frac{\sigma^2}{2}
\exp\left[
-(1+a)a^{-\frac{a}{1+a}}(2k t)^\frac{a}{1+a}
\right].
\end{equation}
Substituting $a\approx 1$, as empirically found in HMNs \cite{Nat-Comm},
leads to Eq.~(\ref{eq:stretched}),
whose square root behavior is confirmed by simulations in Fig. 4d.


\section{Introduction}

Thanks to the recent energy improvement of the LHC machine we are now able to reach the invariant mass TeV scale,
in search for signs of heavy BSM physics.
On the other hand, exploiting the high integrated luminosity that will be collected over the next years,
we are increasing the precision of low invariant mass measurements that we can compare with high accuracy
theoretical calculations, looking for small discrepancies from SM predictions.
In order to be successful, both these approaches require a theoretical effort to keep up with the experimental progresses.
QCD corrections at N(N)LO are now available for many processes. At this level of precision also QED (N)LO effects become
relevant, and they have to be accounted for consistently~\cite{Martin:2014nqa}.
For some processes like di-lepton production a complete description of the QED dynamics is available~\cite{Dittmaier:2009cr}.

In this work we will focus on the effects of the inclusion of tree-level QED diagrams in SM predictions for the di-lepton final state
at the LHC. In LHC collisions protons also interact electromagnetically.
We will evaluate the contribution of photon interactions to the di-lepton final state, in comparison with the dominant
Drell-Yan (DY) quark-initiated production.
The effect of ``quasi-real" photons ($Q^2 \simeq 0$) can be accounted for through the Equivalent Photon Approximation~\cite{Budnev:1974de},
a well known and successful procedure that exploit the dipole approximation to model the proton electromagnetic field.
More recently, PDF collaborations have released QED PDF sets, which include a component of photon within the nucleons.
Effectively in this description the photon is treated as any other parton, and we can use those QED PDFs to explore
the effects of ``real" photons ($Q^2 = 0$), as they now resolved.

The importance of the inclusion of Photon-Initiated (PI) terms have been discussed in the literature concerning different
final states~\cite{Accomando:2016tah, Harland-Lang:2016kog, CarloniCalame:2007cd, Alioli:2016fum, Bourilkov:2016qum, Bourilkov:2016oet, Dyndal:2015hrp, Pagani:2016caq},
and their effects are now accounted for in the experimental analysis~\cite{CMS:2016abv, Aaboud:2016cth}.

\section{Real and virtual photons}
\label{sec:photons}

We are interested in evaluating the effects of diagrams with two photons in the initial state, and two charged leptons in the final state.
The process $\gamma\gamma \rightarrow \ell^+\ell^-$ receive contributions from the $t$- and $u$-channel exchange of
a charged lepton.

As mentioned, QED PDF sets can be invoked to evaluate the contribution of those diagrams.
Some of these PDF sets, such as NNPDF3.0QED~\cite{Ball:2014uwa}, xFitter\_epHMDY~\cite{Giuli:2017oii},
CT14QED\_inc~\cite{Schmidt:2015zda} and LUXqed~\cite{Manohar:2016nzj}, are inclusive, meaning that the elastic component resulting
from interactions involving virtual photons is included, while some other sets, such as CT14QED and
MRST2004QED~\cite{Martin:2004dh} are inelastic, that is the elastic component is subtracted off.
The procedures and some of the assumptions adopted by the various PDF collaborations in the fitting of Deep-Inelastic Scattering (DIS)
and possibly LHC data, can be quite different.
For an exhaustive description of each PDF collaboration procedure we refer to the associated literature.

Inelastic sets can be used to separately evaluate the three terms results from two real photons, one real and one
virtual photons, and two virtual photons interactions, denoted respectively as Double-Dissociative (DD),
Single-Dissociative (SD), and pure EPA terms.
We calculate these terms using~\cite{Accomando:2016ehi}:

\begin{eqnarray}
 \label{eq:integration}
 \frac{d\sigma_{DD}}{dM_{\ell\ell}} & = & \iint dx_1 dx_2 \frac{\left|\mathcal{M}(\gamma\gamma\rightarrow \ell^+\ell^-)\right|^2}{32\pi M_{\ell\ell}} f_\gamma(x_1, Q)f_\gamma(x_2, Q) \label{eq:double_diss}\\
 \frac{d\sigma_{SD}}{dM_{\ell\ell}} & = & \int dQ^2_1\iint dx_1dx_2 \frac{\left|\mathcal{M}(\gamma\gamma\rightarrow \ell^+\ell^-)\right|^2}{32\pi M_{\ell\ell}} N(x_1,Q^2_1)f_\gamma(x_2, Q)+(x_1\leftrightarrow x_2) \label{eq:single_diss}\\
 \frac{d\sigma_{EPA}}{dM_{\ell\ell}} & = & \int dQ^2_1 \int dQ^2_2 \iint dx_1dx_2 \frac{\left|\mathcal{M}(\gamma\gamma\rightarrow \ell^+\ell^-)\right|^2}{32\pi M_{\ell\ell}} N(x_1,Q^2_1)N(x_2,Q^2_2) \label{eq:EPA}
\end{eqnarray}

In eqs.~(\ref{eq:single_diss}) and~(\ref{eq:EPA}), $N(x,Q^2)$ represent the spectrum of virtual photons.
Its derivation is given in~\cite{Budnev:1974de}, while a more recent treatment, including the numerical values for the EPA
parameters that we have used in this analysis, is available in~\cite{Piotrzkowski:2000rx}.
The integration over the virtual momentum $Q^2$ is constrained between $Q^2_{min}$ that is kinematically determined,
and $Q^2_{max}$ that is arbitrary, and taken to be varied around the fixed value at $Q^2_{max} = 2~\rm{GeV}^2$.
Results for the three terms separately in the invariant mass range of few TeV are given in fig.~\ref{fig:PI_EPA_single}
using the MRST2004QED and the CT14QED PDF sets respectively.

\begin{figure}[h]
\begin{center}
\includegraphics[width=0.45\textwidth]{./fig1a.eps}{(a)}
\includegraphics[width=0.45\textwidth]{./fig1b.eps}{(b)}
\caption{Individual photon-induced contributions to the di-lepton spectrum at the LHC@13TeV.
From bottom to top, the dashed black line represent the DY prediction, the light and dark coloured
curves represent the DD and SD results respectively, while the solid black line represents the pure EPA term.
The shaded areas include the systematic uncertainties. The inset plots show the ratio between the SD and the DD results.
We show results for the (a) MRST2004QED set and (b) the CT14QED set.}
\label{fig:PI_EPA_single}
\end{center}
\end{figure}

While the pure EPA contribution appears negligible with respect to the others, the DD and the SD results are of
comparable size. In the inset plots we show their ratio. The SD size results of the order of 35 - 40\% of the DD term
in the MRST2004QED picture, while using the CT14QED set we find that the SD size varies between 75\% and 90\% of the
DD result in the examined invariant mass region.

In the plots we are also showing the systematic uncertainties, represented by the shaded areas around each curve.
The systematics coming from the virtual photon spectrum have been evaluated fixing different values for the $Q^2_{max}$
in the integrations of eqs.~(\ref{eq:single_diss}) and~(\ref{eq:EPA}), respectively $Q^2_{max} = 0.5~\rm{GeV}^2$ and
$Q^2_{max} = 8~\rm{GeV}^2$ for the lower and upper error band.
The systematic uncertainties for the real photon spectrum have been calculated following the specific PDF collaboration
adopted procedure. In particular for MRST2004QED we do not have any prescription to evaluate the uncertainties, while
the CT14QED set is accompanied by a table of 31 PDFs, each one imposing a progressive constrain on the fraction of
total momentum carried by the photon. From their analysis the upper bound is set to be $p_\gamma \leq 0.11\%$
at 68\% C.L.~\cite{Schmidt:2015zda} and following this result we have extracted central value
and error band for the real photons contributions.

\section{Inclusive results and PDF uncertainties}
\label{sec:inclusive_and_systematics}
The sum of the EPA, SD and DD terms can be directly compared with the result obtained by evaluating the DD integration
of eq.~\ref{eq:double_diss} using inclusive sets, since they already combine both the elastic and inelastic components.
In order to verify that the separation of the various terms has been done correctly, we have compared the sum of the DD, SD and EPA
results obtained with the CT14QED set with the inclusive result from the CT14QED\_inc set.
The result is visible in fig.~\ref{fig:PI_complete}(a) where we have plotted the ratio of those two results (blue line).
The two results are in good agreement, the differences being $\leq 3\%$. This ensures that double counting
effects are well under control.
In the same plot we show the comparison between two predictions for the inclusive PI results obtained with the LUXqed and the
CT14QED\_inc sets. The two central values are in good agreement, as their difference is always $\leq 7\%$.
In fig.~\ref{fig:PI_complete}(b) we give the inclusive PI results for the various PDF sets (coloured lines)
in comparison with the DY expectations (black line).
The shaded areas represent the systematic uncertainties on the PI predictions.

\begin{figure}[h]
\begin{center}
\includegraphics[width=0.45\textwidth]{./fig2a.eps}{(a)}
\includegraphics[width=0.45\textwidth]{./fig2b.eps}{(b)}
\caption{(a) Comparison between the full PI prediction from the CT14QED\_inc set and the LUXqed, and between the CT14QED\_inc result and the sum of the EPA, DD and SD terms obtained with the CT14QED set.
(b) Full PI prediction for different PDF sets (both inelastic and inclusive) in comparison with the pure DY term.}
\label{fig:PI_complete}
\end{center}
\end{figure}

The first thing to notice is the large difference in the error band predictions. The latter have been estimated following the
PDF collaboration prescriptions, as already discussed for the CT14QED case.
The NNPDF3.0QED prescription follows the ``replicas" method, in this case 100 replicas contain the information on the
systematic uncertainties.
The LUXqed set has been released in the PDF4LHC delivery~\cite{Butterworth:2015oua}, along with a set of 100 symmetric Hessian
eigenvectors to calculate the error.

\begin{figure}[h]
\begin{center}
\includegraphics[width=0.45\textwidth]{./fig3a.eps}{(a)}
\includegraphics[width=0.45\textwidth]{./fig3b.eps}{(b)}
\caption{(a) Relative size of the photon PDF uncertainties over the complete di-lepton result.
Note the scale factor associated to some of the curves associated to the results obtained different QED PDF sets.
(b) The di-lepton spectrum, sum of the DY and the PI central values and its PDF uncertainties as predicted by different QED PDF sets.}
\label{fig:errors_dilepton}
\end{center}
\end{figure}

The size of the errors that we have obtained varies in a wide range.
The result is shown is fig.~\ref{fig:errors_dilepton}(a) where the three curves have different scale factors, as visible in the legend.
The most optimistic results are given by the LUXqed set, where the PDF uncertainty is of the order of 1\% along all the spectrum,
and the central value of the PI contribution is rather small with respect to the DY term.
The most conservative scenario is given by the NNPDF3.0QED set, where the size of the systematics can be one order of magnitude larger
than the predicted central value, which is rather large for this set.
An intermediate picture is given by the CT14QED\_inc set, where the PDF errors are between 20\% and 30\% along the spectrum.

The sum of the DY and PI central values and PDF uncertainties is given in fig.~\ref{fig:errors_dilepton}(b), where some discrepancies
on the shape of the distribution are visible at high invariant masses.

\section{PI effects on BSM searches in the di-lepton channel}
\label{sec:BSM}

The results we have shown demand some caution when dealing with the interpretation of experimental analysis.
From here on we will consider the most conservative scenario given by the NNPDF3.0QED set.
The large uncertainties (and high central value) that we obtain adopting this PDF set, have considerable effect on the theoretical predictions
that are to be compared with the experimental data.

Especially in the high invariant mass region, the systematics show a large uncertainty on the cross section predictions.
The integrated cross section in particular receive a sizeable contribution from the PI processes inclusion.
If we consider the upper limit of the 1$\sigma$ error band, the integrated result differs up to two orders of magnitude with respect to the
pure DY prediction. This is shown in fig.~\ref{fig:integrated_events}(a), with the two results as in the legend.

\begin{figure}[h]
\begin{center}
\includegraphics[width=0.45\textwidth]{./fig4a.eps}{(a)}
\includegraphics[width=0.45\textwidth]{./fig4b.eps}{(b)}
\caption{(a) Integrated cross section as function of the low invariant mass cut in the integration for the di-lepton channel, with and without
the inclusion of the PI central value and PDF uncertainties.
(b) Expected number of events and its statistic and systematic error from PDF at different luminosities,
as function of the low invariant mass cut in the integration.
NNLO QCD corrections to Drell-Yan production~\cite{Hamberg:1990np, Harlander:2002wh} have been included according to the
method detailed in~\cite{Accomando:2016tah, Accomando:2016ehi, Accomando:2015cfa, Accomando:2016mvz},
as well as the declared experimental acceptance and efficiency factors for the electron channel~\cite{Khachatryan:2014fba}.}
\label{fig:integrated_events}
\end{center}
\end{figure}

Considering the high integrated luminosity that will be collected over the years, we will soon expect to register some high invariant mass events.
For their correct interpretation, the inclusion of PI processes and uncertainties in the analysis is crucial.
As visible in fig.~\ref{fig:integrated_events}(b), because of the uncertainty on the spectrum at high invariant masses, we can observe
events even above 5~TeV invariant mass.

We want to explore more in detail the effects of these uncertainties in BSM searches for heavy resonances in the di-lepton channel.
Neutral heavy resonances naturally appear in a variety of BSM constructions and the two leptons final state is the
golden channel for the detection of their decays in collider experiments.
Here we have chosen two popular benchmarks~\cite{Accomando:2010fz} to study the consequences of our previous results on BSM $Z^\prime$-boson searches.
Current limits on the masses of these objects are set at few TeV~\cite{CMS:2016abv, Aaboud:2016cth}.

\begin{figure}[h]
\begin{center}
\includegraphics[width=0.45\textwidth]{./fig5a.eps}{(a)}
\includegraphics[width=0.45\textwidth]{./fig5b.eps}{(b)}
\caption{(a) Differential cross section and (b) $AFB^*$ distribution for a heavy $Z^\prime$ with 3.5~TeV mass as predicted by the
$E_6^\chi$ model. The shaded areas represent the 1$\sigma$ PDF error band.}
\label{fig:E6_chi}
\end{center}
\end{figure}

To probe the narrow $Z^\prime$ case, we have considered the $E_6^\chi$. As visible in fig.~\ref{fig:E6_chi}(a) the Breit-Wigner peaked shape
stands well above the uncertainties, thus bump searches for resonant objects do not seem to be much affected by photon PDFs errors
in the invariant mass range under exam.
A combined analysis with the inclusion of another observable, like the Forward-Backward Asymmetry (AFB), could help in the interpretation of the results.
In the AFB observable systematics uncertainties are indeed significantly reduced~\cite{Accomando:2016ehi, Accomando:2015cfa}, thus a deviation from the flat SM predictions
would be a clear sign of new physics. We give an example for a narrow resonance signal in fig.~\ref{fig:E6_chi}(b), again for the $E_6^\chi$ benchmark.
However, even if the large systematics from the PI result are partially cancelled, the inclusion of those terms has an effect on the SM predictions.
The PI being an angularly symmetric process, has the effects of reducing the overall AFB, thus their inclusion is essential for the correct
extrapolations of SM background.

\begin{figure}[h]
\begin{center}
\includegraphics[width=0.45\textwidth]{./fig6a.eps}{(a)}
\includegraphics[width=0.45\textwidth]{./fig6b.eps}{(b)}
\caption{(a) Differential cross section and (b) $AFB^*$ distribution for a heavy $Z^\prime$ with 3~TeV mass as predicted by the
$GSM-SSM$ model. The width of the resonance has been fixed to 20\% of its mass. The shaded areas represent the 1$\sigma$ PDF error band.}
\label{fig:SSM}
\end{center}
\end{figure}

With the purpose of exploring the PI effects on non-resonant searches, we are here considering another popular BSM model featuring a $Z^\prime$-boson,
the $GSM-SSM$ model, where the resonance width is enhanced.
In fig.~\ref{fig:SSM}(a) we show the invariant mass profile of a $Z^\prime$-boson with fixed width over mass ratio at 20\%.
This picture is quite representative of many other scenarios, like contact interactions, extra dimensions~\cite{Accomando:1999sj, Accomando:2016mvz},
or continuous spectra of BSM resonances,
as in the ADD model~\cite{Mimasu:2011sa}.
As visible, now the error bands clearly reduce our sensitivity to this kind of signal. As well the typical shape of a non-resonant object
resulting similar to a shoulder, can be confused with the tail of PI processes contribution.
Again, in this context the AFB observable can be used to corroborate the interpretations, since a typical $Z^\prime$ signal would maintain its
shape even in the wide resonance scenario, and again the uncertainties from photon PDFs are here reduced.

\section{Conclusions}
\label{sec:conclusions}

We have discussed the contributions of PI processes to the di-lepton production channel at the LHC.
The contributions of the new interactions have been obtained separating the effects of ``quasi-real" and ``real" photons, and they have been
treated adopting the EPA and with the use of QED PDFs respectively.
We have computed the central values of the PI terms using different QED PDF sets and when available we have also estimated the PDF uncertainties
along following the appropriate prescriptions.

We have discussed those results and compared with the dominant DY contribution.
In particular at high invariant masses, deviations from the pure DY predictions can occur.
The size of these deviations, and the associated theoretical systematics, vary significantly with the different scenarios for the photon PDFs.
We have analysed the sensitivity of BSM searches for both resonant and non-resonant objects, in light of the
previous results, adopting the most conservative QED PDF set.

Bump searches for resonant objects that follow a peaked Breit-Wigner shape are not much affected by photon interactions, while counting experiments
for non-resonant objects would suffer a significant loss of sensitivity.
The interpretation of experimental data can be supported by introducing an extra observable as the Forward-Backward Asymmetry, particularly because
of its favourable features concerning systematic uncertainties.

\acknowledgments
We thank A. Belyaev and A. Pukhov for discussions on the EPA. We are grateful to V. Bertone for discussions on QED PDFs and for his help with NNPDF parton distribution calculations.
This work is supported by the Science and Technology Facilities Council, grant number ST/L000296/1.
F. H. acknowledges the support and hospitality of DESY, the University of Hamburg and
the DFG Collaborative Research Centre SFB 676 ``Particles, Strings and the Early Universe".
All authors acknowledge partial financial support through the NExT Institute.

\bibliographystyle{apsrev4-1}


\section{Conclusions}\label{sec:conclude}

We have presented here a new method, GrannGAN, for implicit distribution learning and generating features of nodes and edges conditioned on graph skeletons.
The promising results of the method, as documented in our experiments, indicate that the method can learn the complex high-dimensional distributions and generate new data examples with features coherent with the underlying graph structure.
When the generation of new skeletons is not crucial, GrannGAN can be used as a full generative model by sampling the skeleton from the data.
It is also the first method for graph generation conditioned on the skeleton.

These favorable results provide a solid starting point for multiple possible extensions in future work.
For example, in some applications (e.g. bio-chemistry) a reasonable starting point for the feature generations may be a partly annotated graph (in place of a completely non-annotated graph).
Our model, which is already conditional, could be easily adapted to this setting by re-using the ideas of our second edge-annotation phase.
Another interesting extension is that of property optimization.
An additional conditioning variable could be included in the generators and critics to guide the model in generating property-conditioned features.

\section{Experiments}\label{sec:experiments}

In this section, we present experiments using our model as a graph generative model.
There currently exists no other models conditioning on the graph skeleton and, therefore, no other method to directly compare with.
Instead, we use generative models presented in the previous section as baselines.
However, we underline that these models do not have the same purpose.
In the second part of this section, we also present the results of the conditional generation by fixing the skeleton.

We evaluate our method in a set of experiments over three datasets containing (node- and edge-)  annotated graphs\footnote{Unfortunately, we did not found other public dataset with enough node- and edge-annotated graphs.}.
Two of the datasets, QM9 \cite{ramakrishnan_quantum_2014} and ZINC \cite{sterling_zinc_2015}, are from the chemical domain that is frequently used as a test bed for annotated graph modeling.
The third is the fingerprint dataset \cite{Riesen2008} included in the TUDataset collection \cite{Morris+2020}.

As it has recently been discussed, for example in \cite{obray_evaluation_2021} and \cite{thompson_evaluation_2021}, evaluating generative models of graph structures is particularly challenging due to the impracticality of visual or perceptual comparisons of the data examples. The comparison with existing graph generative models is therefore difficult and can provide only crude indication of the method capabilities.

For the chemical datasets the most common evaluation metrics are the following
\begin{itemize}
\item \emph{validity} is the proportion of chemically valid molecules in the total generated examples and measures the ability of the method to understand the chemical constraints differentiating general graphs from chemically valid molecules
\item \emph{uniqueness} is the proportion of unique samples within the \emph{valid} generated samples and measures the variability of the generated data
\item \emph{novelty} is the proportion of examples within the \emph{valid} and \emph{unique} set that do not exist in the training dataset and measures the ability of the method to go beyond data memorization
\end{itemize}
We complement these by an \textbf{overall} score of \emph{valid-unique-novel} molecules calculated simply as the product of the three above metrics and measuring the overall quality of the generated examples as a proportion of samples with all three of these desirable properties in the set of generated data. So, the overall score gives the rate of valid molecules that are neither in the dataset nor already generated. Note that in most cases the overall score is the metric of interest.

While these evaluation metrics may be useful indicators for chemical downstream tasks, they do not provide any measure of the distance between the distribution of reference and the generated distributions.
Various indicators calling on the framework of maximum mean discrepancy (MMD) \cite{gretton_kernel_2012} have been proposed in the literature.
These are used very inconsistently and, as demonstrated in \cite{obray_evaluation_2021}, they are highly sensitive to the specific choice of the graph statistics, the kernel and the hyperparameters used for the MMD calculation.
Instead of calculating the MMDs on graphs statistics, we report directly the distance between the generated feature distributions, using the Jenson-Shannon Distance for each feature. We show that with respect to these metrics, our model outperforms by far GraphAF, model considered as the state-of-the-art.

\subsection{QM9}\label{sec:qm9}

QM9 is one of the most commonly used datasets for testing models for annotated graph generation.
It consists of $\sim$134k stable organic molecules with up to 9 atoms of four types.
The 4 atom types and 4 bond types are encoded as one-hot vectors in the node feature descriptions ${\mathbf{v}}$.

In table \ref{tab:qm9} we present an overview of the generative performance of our GrannGAN method in comparison with a set of well-established graph generative methods: MolGAN \cite{de_cao_molgan_2018}, GraphVAE \cite{simonovsky_graphvae_2018}, GraphNVP \cite{madhawa_graphnvp_2019},
GraphAF \cite{shi_graphaf_2019}, CharacterVAE \cite{gomez-bombarelli_automatic_2018}, GrammarVAE \cite{kusner_grammar_2017}, and JT-VAE \cite{jin_junction_2018}.

The GrannGAN results are calculated from 1000 new data examples generated by conditioning on skeletons randomly sampled from the training data.
The results of the baseline methods are those reported by the authors in the original papers\footnote{CharacterVAE and GrammarVAE results (not reported in the original papers) are taken over from the method replications in GraphNVP.}.
Despite the frailty of such comparisons due to inconsistencies in the original experimental protocols (for example, some of the methods, such as GraphAF, work over the \emph{kekulized} versions of the molecules reducing the number of edge categories to three), our GrannGAN achieves excellent results. It outperforms all one-go generative models and is on par with the best auto-regressive model.

\setlength\tabcolsep{5pt}
\begin{table}[htp]
\begin{center}
        \caption{QM9: performance comparison}\label{tab:qm9}
        \begin{tabular}{ c  l | c c c | c }
        \hline
        & \textbf{Model} &  \textbf{valid} & \textbf{unique} & \textbf{novel} & \textbf{overall}   \\
        \hline \hline

        \rowcolor{lgray}
        \multirow{5}{*}{\begin{sideways} no chemistry \end{sideways}} & GrannGAN & 82.5 & 99.9 & 64.9 & 53.4 \\
        & MolGAN (wo. RL)  & 87.7 & 2.9 & 97.7 & 2.5 \\
        & GraphVAE  & 55.7 & 76.0 & 61.6 & 26.1 \\
        & GraphNVP & 83.1 & 58.2 & 99.2 &  46.8 \\

        & GraphAF (wo. validity) & 67.0 & 94.5 & 88.8 & 56.3 \\
        \hline \hline
        \multirow{5}{*}{\begin{sideways} \gtext{ chemistry} \end{sideways}} & \gtext{MolGAN-RL} & \gtext{99.8} & \gtext{2.3} & \gtext{97.9} & \gtext{2.2} \\
        & \gtext{CharacterVAE} & \gtext{10.3} & \gtext{90.0} & \gtext{67.5} & \gtext{11.9} \\
        & \gtext{GrammarVAE} & \gtext{60.2} & \gtext{80.9} & \gtext{9.3} & \gtext{11.9} \\
        & \gtext{GraphAF} & \gtext{100} & \gtext{94.5} & \gtext{88.8} & \gtext{83.9} \\
        \hline
        \end{tabular}
\end{center}
\end{table}

As explained above, the metrics in the table are very much domain specific.
Methods in the lower part of table \ref{tab:qm9} expressly focus on these
introducing into the models various expert-designed modules promoting chemical validity of the generated molecular graphs (e.g., valid sub-substructures vocabulary, valency constraints, rejection sampling, etc.).
While these improve the targeted metrics in table \ref{tab:qm9}, they also bias the generative process and, therefore, the model distribution towards the validity metric in exchange for the distribution approximation goal.
We list these methods in table \ref{tab:qm9} for completeness despite them not being really comparable to our approach which focuses on the distribution approximation of general graphs.

\begin{figure}[htp]
\centering
\includegraphics[width=0.3\textwidth]{pics/qm9_1}
\includegraphics[width=0.3\textwidth]{pics/qm9_2}\\
\includegraphics[width=0.3\textwidth]{pics/qm9_3}
\includegraphics[width=0.3\textwidth]{pics/qm9_4}
\caption{QM9: empirical distributions of real and generated features  (samples of 1000 graphs)}
\label{fig:qm9}
\end{figure}

\setlength\tabcolsep{5pt}
\begin{table}[htp]
\begin{center}
        \caption{Jenson-Shannon distances between real and generated distributions for node and edge features of the QM9 and ZINC datasets (the lower the better)}\label{tab:JSmolecule}
        \begin{tabular}{ l | c c c c}
        \hline
        \textbf{Model} &  \textbf{nodes - QM9} & \textbf{edges - QM9} &  \textbf{nodes - ZINC} & \textbf{edges - ZINC}    \\
        \hline \hline

        \rowcolor{lgray}
        GrannGAN & $1.87 \cdot 10^{-3}$ & $ 0.04 \cdot 10^{-4}$ & $1.50 \cdot 10^{-3}$ & $0.60 \cdot 10^{-3}$\\
        GraphAF  & $15.80 \cdot 10^{-3}$ & $11.5 \cdot 10^{-4}$ & $15.5 \cdot 10^{-3}$ & $3.39 \cdot 10^{-3}$ \\

        \hline
        \end{tabular}
\end{center}
\end{table}

Figure \ref{fig:qm9} documents the ability of the GrannGAN method to learn the feature distribution by comparing the empirical real data distribution with the distribution of the generated examples.
We have replicated the results of the GraphAF method, which is the most competitive with GrannGAN according to table \ref{tab:qm9}), to extract similar statistics.
GrannGAN matches the feature distributions of the reference dataset almost exactly while we observe a less closer match in the GraphAF\footnote{GraphAF uses somewhat different training set than we. For example, by kekulizing the molecules it removes all aromatic edges and thus reduces the number of edge types to 3. This is true also for the ZINC dataset.} generated data.
This observation is numerically confirmed in table \ref{tab:JSmolecule} listing the Jensen-Shannon distances (JSD) between the real and generated data samples, where GrannGAN is systematically better by one order of magnitude.

\subsection{ZINC}\label{sec:zinc}

We use here the ZINC250k version used previously for generative modeling evaluations.
It consists of 250k randomly selected molecules from the complete ZINC set with up to 38 atoms of 9 types and 4 bond types.

Table \ref{tab:zinc} summarizes the generative performance of GrannGAN over 1000 newly synthesized examples compared to results reported in the literature.
MolGAN and GraphVAE did not experiment (or did not provide the results) on this more challenging dataset of larger graphs.
GrannGAN scales to this dataset easily and performs competitively with respect to the chemically motivated metrics (keeping in mind the difficulties of such comparisons related to differences in experimental protocols and evaluation procedures).

Figure \ref{fig:zinc} documents the excellent performance of GrannGAN in learning the feature distributions, matching it closely for both the node and edge features.
The differences in the true and GraphAF generated data distributions are more pronounced as is also clear from the numerical evaluation via the JSD presented in table \ref{tab:JSmolecule}.

\setlength\tabcolsep{5pt}
\begin{table}[htp]
\begin{center}
        \caption{ZINC: performance comparison}\label{tab:zinc}
        \begin{tabular}{ c  l | c c c | c }
        \hline
        & \textbf{Model} &  \textbf{valid} & \textbf{unique} & \textbf{novel} & \textbf{overall}   \\
        \hline \hline

        \rowcolor{lgray}
        \multirow{5}{*}{\begin{sideways} no chemistry \end{sideways}} & GrannGAN & 56.5 & 100 & 100 & 56.5 \\
        & MolGAN (wo. RL)  & --- & --- & --- & --- \\
        & GraphVAE  & 13.5 & --- & --- & --- \\
        & GraphNVP & 42.6 & 94.8 & 100 &  40.4 \\

        & GraphAF (wo. validity) & 68.0 & 99.1 & 100 & 67.3 \\
        \hline \hline
        \multirow{5}{*}{\begin{sideways} \gtext{chemistry} \end{sideways}} & \gtext{MolGAN-RL} & \gtext{---} & \gtext{---} & \gtext{---} & \gtext{---} \\
        & \gtext{CharacterVAE} & \gtext{7.2} & \gtext{9.0} & \gtext{100} & \gtext{0.6} \\
        & \gtext{GrammarVAE} & \gtext{0.7} & \gtext{67.5} & \gtext{100} & \gtext{0.5} \\
        & \gtext{JT-VAE} & \gtext{100} & \gtext{100} & \gtext{100} & \gtext{100} \\
        & \gtext{GraphAF} & \gtext{100} & \gtext{99.1} & \gtext{100} & \gtext{99.1} \\
        \hline
        \end{tabular}
\end{center}
\end{table}

\begin{figure}[htp]
\centering
\includegraphics[width=0.3\textwidth]{pics/zinc_1}
\includegraphics[width=0.3\textwidth]{pics/zinc_2}
\\
\includegraphics[width=0.3\textwidth]{pics/zinc_3}
\includegraphics[width=0.3\textwidth]{pics/zinc_4}
\caption{ZINC: empirical distributions of real and generated features (samples of 1000 graphs)}
\label{fig:zinc}
\end{figure}

\subsection{Fingerprints}\label{sec:finger}

Fingerprint is a dataset of graphs representing the skeletonized regions of interest in human fingerprints \cite{Riesen2008}.
The graphs consist of up to 26 nodes.
Both nodes and edges are described by 2 continuous features related to their position and orientation in the 2d fingerprint image.
We have linearly re-scaled both node and edge features between -1 and 1.

We use the non-annotated graphs as the skeletons for new feature generations.
Each row displays graphs arranged according to the new features generated by GrannGAN.

\begin{figure}[htp]
\centering
\includegraphics[width=0.25\textwidth]{pics/v1.png}
\includegraphics[width=0.25\textwidth]{pics/v2.png}
\includegraphics[width=0.25\textwidth]{pics/d.png}
\caption{Fingerprints: empirical distributions ${\mathbf{v}}_1$, ${\mathbf{v}}_2$ and ${\mathbf{d}}$ sampled from our GrannGAN and from the dataset  (samples of 1000 graphs)}
\label{fig:FP_node}
\end{figure}

\begin{figure}[htp]
\centering
\includegraphics[width=0.3\textwidth]{pics/e1.png}
\includegraphics[width=0.3\textwidth]{pics/e2.png}

\caption{Fingerprints: empirical distributions ${\mathbf{e}}_1$ and ${\mathbf{e}}_2$ sampled from our GrannGAN and from the dataset  (samples of 1000 graphs)}
\label{fig:FP_edge}
\end{figure}

In table \ref{tab:fingerprint} we list the JS distance (with the logarithm in base 2) between the empirical distributions of the training data and newly generated features starting from 500 randomly selected skeletons.
These have been calculated by discretizing the features into 200 even sized bins.
The columns in the table correspond to the two node ${\mathbf{v}}$ and edge ${\mathbf{e}}$ features respectively.
Following the same methodology, we also report the JS distance in the distribution of the euclidean distances between connected nodes ${\mathbf{d}}$.
Unlike the feature distribution, this statistic consider the relation between node features.
In figures \ref{fig:FP_node} and \ref{fig:FP_edge}, we show the node and edge features distribution for the data and newly generated examples from our model as well as the distribution of distances between node ${\mathbf{d}}$\footnote{We assume that the oscillation of the curve is just an artifact coming from the relatively small data points by bin}.

\setlength\tabcolsep{5pt}
\begin{table}[htp]
\begin{center}
        \caption{Jenson-Shannon distances between real and generated distributions for node and edge features of the Fingerprint dataset (the lower the better)}\label{tab:fingerprint}
        \begin{tabular}{c c c c c c}
        \hline
        &\textbf{v1} &  \textbf{v2} & \textbf{d} & \textbf{e1} & \textbf{e2} \\
        \hline
        \textbf{GrannGAN} & $5.02 \cdot 10^{-2}$ &  $4.55 \cdot 10^{-2}$ & $32.84 \cdot 10^{-2}$ & $16.78 \cdot 10^{-2}$ & $9.74 \cdot 10^{-2}$ \\
        \hline
        \textbf{Data samples} & $1.13 \cdot 10^{-2}$ &  $1.27 \cdot 10^{-2}$ & $1.49 \cdot 10^{-2}$ & $1.37 \cdot 10^{-2}$ & $1.07 \cdot 10^{-2}$ \\
        \hline
        \end{tabular}
\end{center}
\end{table}

Again, our model shows excellent performance in closely matching the node and edge features and their relations.

These experiments show the quality of our GrannGAN for the generative task.
It outperforms all previous one-go generative models.
The sequential GraphAF is the only model presenting slightly better results using the molecular metrics.
However, we show that our model is much better at capturing the feature distributions.

\subsection{Conditional generation}
In this section, we show that our model can be used for conditional generation by fixing the skeleton and still producing a high level of diversity.

We want to guarantee that the sample diversity does not come mainly from the skeleton. To test the generative capacity of our GrannGAN, we randomly sampled 100 skeletons from the ZINC dataset. We generate 1000 new instances conditioned on each sampled skeleton. We compute the average rate of uniqueness, i.e. the rate of unique instances over all the valid molecules.

Table \ref{tab:condgen} reports the results of the experiment.
As expected, the validity rate is similar to the one by sampling a new skeleton for each instance.
Even by fixing the skeleton, we reached a rate of 92.5\% unique instances on average.
On average, the conditional generation of 1000 instances over the same skeleton produces 527.0 unique molecules.
So, we show that the performance of our model does not depend critically on the sampling of multiple skeletons and can be properly used as a conditional generative model.

\setlength\tabcolsep{5pt}
\begin{table}[htp]
\begin{center}
        \caption{Conditional generation of 1000 instances from the same skeleton (average)}\label{tab:condgen}
        \begin{tabular}{c | c c c}
        \hline
        \textbf{Model} &  \textbf{Validity (\%)} &  \textbf{Uniqueness (\%)} & \textbf{Valid and unique (\textperthousand)} \\
        \hline
        \textbf{GrannGAN} & $57.0$ &  $92.5$ & $527.0$  \\

        \hline
        \end{tabular}
\end{center}
\end{table}

We have demonstrated that the GrannGAN is competitive with state-of-the-art model for graph generation even though it is first meant for conditional generation.
GraphAF is slightly better on the molecular metric, but much worst at matching the feature distribution.
In addition, our model generates instances in one go and is therefore much faster during inference.
More fundamentally, our model preserves its quality while generating new instances conditioned on the skeleton, a task that no other model can do.

\section{Introduction}\label{sec:intro}

Modern deep learning approaches for learning high-dimensional data distributions and synthesizing new data examples have achieved great successes in a  number of domains.
We focus here on the particularly challenging problem of generating new data examples ${\mathbf{x}}_i \in \mathbb{R}^{m_i}$ with different dimensionalities $m_i$ between individual instances and complex relational structures in the feature spaces organized in graphs.

More concretely, we focus on learning the distributions and generating new examples of features associated with nodes and edges of annotated graphs conditioned on the graph structure.

We formulate the problem as that of modeling a conditional distribution, where we learn to generate the node and edge features conditionally on the given graph skeleton (non-annotated graph).
In this respect, our paper complements the existing tool-set of models for non-annotated graph generations (e.g. \cite{you_graphrnn_2018,liao_efficient_2019}).
The non-annotated graph can be generated by one of those or, more interestingly, can be provided by the user based on the needs of a downstream task that requires a specific graph structure (e.g. basis for scaffold-based \emph{de novo} drug discovery or molecular docking, types and interactions of particles in high energy physic, individual features and types of relations in a social network).

The graph annotation generative adversarial networks (GrannGAN) method that we propose follows the strategy of implicit distribution modeling via adversarial training \cite{goodfellow_generative_2014}, allowing us to sample new graphs from a distribution $p$ approximating the true $p^*$ without explicitly formulating the $p$ distribution function.
In principle, the method can generate node of edge features independently.
In practice, we generate the node and edge features in two phases depicted in Figure \ref{fig:archi}.
In the first phase, our model samples the node features conditioning on the graph skeleton. In the second phase, it uses the generated node features as an additional conditional variable to sample the edge features.

The entire model architecture relies on the permutation equivariant transformations of message passing neural networks (MPNN) \cite{gilmer_neural_2017}.
Thanks to these, we circumvent the difficulties related to the ordering of graph objects in their representation.
This critical property of the method allows for efficient use of the model capacity, which needs to learn neither a particular heuristic for unique representation ordering (as in the case of linearized representations) nor the complete set of equivalent permutations.

In the following, we describe the newly proposed GrannGAN method and provide some details of the technical implementation in sections \ref{sec:method}.
We then position the method within the existing state of research and document its performance competitive with the best of the well-established methods for graph generations on a set of experiments in section \ref{sec:related} and \ref{sec:experiments}.
We conclude with a discussion of possible future directions in section \ref{sec:conclude}.

\section{GrannGAN}\label{sec:method}

Let ${\mathcal{G}} = \{ {\mathcal{V}}, {\mathcal{E}}, V, E \} $ be an undirected graph with a set of nodes (vertices) ${\mathcal{V}}$ and a set of edges ${\mathcal{E}}$ between pairs of nodes in the graph, $V$ and $E$ are the corresponding node and edge features.
Let $\nu_i$ denote a node $i$ and ${\mathbf{v}}_i \in \mathbb{R}^d$ the features associated with that node. An edge $\epsilon_{ij}$ is connecting the pair of nodes $\nu_i$ and $\nu_j$ and ${\mathbf{e}}_{ij} \in \mathbb{R}^c$ are features corresponding to that edge.
We consider only the case where the edges are undirected so that $\epsilon_{ij}= \epsilon_{ji}$ and ${\mathbf{e}}_{ij} = {\mathbf{e}}_{ji}$, but the model can be easily extended to the case with directed edges.
We further use the term \emph{skeleton} and the letter ${\mathcal{S}}$ to refer to the non-annotated graph ${\mathcal{S}} = \{{\mathcal{V}}, {\mathcal{E}}\}$ corresponding to ${\mathcal{G}}$.

\subsection{Model factorization}\label{sec:factorization}

In our approach, we model the underlying graph distribution in the following factorization
\begin{align}\label{eq:factorization}
p({\mathcal{G}}) = p({\mathcal{S}}, V, E) = p({\mathcal{S}}) p(V \rvert {\mathcal{S}}) p(E \rvert {\mathcal{S}}, V) \enspace .
\end{align}

Our model consists of modeling $p(V \rvert {\mathcal{S}})$ and $p(E \rvert {\mathcal{S}}, V)$ with conditional GAN. Note that we could equivalently reverse the conditioning order and model $p(E \rvert {\mathcal{S}})$ and $p(V \rvert {\mathcal{S}}, E)$.
In our experiments, we found the first option yielding better results. So, we keep this ordering for the following.

During training, we sample the graph skeleton from the data distribution $p^*({\mathcal{S}})$. At inference, we can either sample the graph skeleton or keep it fixed depending on the task.

\subsection{Implicit data generation}\label{sec:cGAN}

We use the Wasserstein-GAN (WGAN) \cite{arjovsky_wasserstein_2017} formulation of adversarial training with spectral normalization \cite{miyato_spectral_2018} in all linear layers of the critic to enforce 1-lipschitzness.
To sample new examples of data ${\mathbf{x}}$ from a model distribution $p_\theta({\mathbf{x}})$ WGAN uses a generator $g_\theta$ mapping from a random latent variable ${\mathbf{z}}$ to the output ${\mathbf{x}} = g_\theta({\mathbf{x}})$.
The generator is learned to minimize the Wasserstein-1 (or Earth-Mover) distance  $W(p_\theta({\mathbf{x}}), p^*({\mathbf{x}}))$ between the implicit model distribution and the true generative distribution through a min-max optimization.

\begin{align}\label{eq:WGAN}
\min_\theta \max_\varphi \ \mathbb{E}_{(y) \sim p^*} f_\varphi({\mathbf{y}}) - \mathbb{E}_{z \sim p(z)} f_\varphi\big(g_\theta({\mathbf{z}})\big) \enspace ,
\end{align}
where $f_\varphi$ is the K-Lipschitz critic function mapping from the data ${\mathbf{y}}$ (real or generated) to a real-valued score $f_\varphi({\mathbf{y}}) \in \mathbb{R}$.

We use the conditional WGAN formulation twice in the GrannGAN pipeline sketched out in figure \ref{fig:archi}.
Following the factorization in equation \eqref{eq:factorization} we first model the conditional distribution $p(V \rvert {\mathcal{S}})$ in a \emph{node-annotation} phase.
Here, the method generates the node features $\hat{V} = g_{V, \theta}({\mathcal{Z}}, {\mathcal{S}})$ from the set of latent noise variables ${\mathcal{Z}}$ and the skeleton ${\mathcal{S}}$. So, equation \ref{eq:WGAN} becomes
\begin{align}
    \min_\theta \max_\varphi \ \mathbb{E}_{(V, {\mathcal{S}}) \sim p^*} f_{V, \varphi}(V,  {\mathcal{S}}) - \mathbb{E}_{z \sim p(z), {\mathcal{S}} \sim p^*} f_{V, \varphi}\big(g_{V, \theta}({\mathcal{Z}}, {\mathcal{S}}),  {\mathcal{S}} \big) \enspace .
\end{align}

Similarly in the following \emph{edge-annotation} phase, we model the conditional $p(E \rvert {\mathcal{S}}, V)$.
The edge features $\hat{E} = g_{E, \theta}({\mathcal{Z}}, V, {\mathcal{S}})$ are generated conditionally on the previously generated node features by including these as an additional input variable to the generator and critic. In this phase, equation \ref{eq:WGAN} becomes

\begin{align}
    \min_\theta \max_\varphi \ \mathbb{E}_{(E, V, {\mathcal{S}}) \sim p^*} f_{E, \varphi}(E, V, {\mathcal{S}}) - \mathbb{E}_{z \sim p(z), (V, {\mathcal{S}}) \sim p^*} f_{E, \varphi}\big(g_{E,\theta}({\mathcal{Z}}, V, {\mathcal{S}}), V, {\mathcal{S}} \big) \enspace ,
\end{align}

Note that the conditioning on the skeleton ${\mathcal{S}} = ({\mathcal{V}}, {\mathcal{E}})$ acts on the computational graph of the critic $f$ and of the generator $g$, i.e. on the structure of the Message Passing Neural Network (MPNN, explained in detail in the section \ref{sec:MPNN}).
This special way of conditioning through the computational graph is one of the novelties of our method.

\subsection{Implementation}\label{sec:Implementation}

In practice, we start by sampling the skeleton ${\mathcal{S}} = \{{\mathcal{V}}, {\mathcal{E}}\}$ from the data.
For all node $\nu_i \in {\mathcal{V}}$, we sample a random noise vector ${\mathbf{z}}_i$.
So, we get ${\mathcal{Z}} = \{{\mathbf{z}}_i\}_1^n$ and, by construction $|{\mathcal{V}}| = |{\mathcal{Z}}|$.
Doing so, we obtain a graph with random noise vectors as initial latent representation of the node features ${\mathcal{G}}_0 = \{{\mathcal{V}}, {\mathcal{E}}, {\mathcal{Z}}\}$ (and without edge feature).
This graph is the input of the \emph{node-annotation} generator.
Similarly, for the edge generation, we sample a graph from the data without edge feature.
We also sample ${\mathcal{Z}} = \{{\mathbf{z}}_i\}_1^m$, where $m = |{\mathcal{E}}|$.
The input of the \emph{edge-annotation} generator is the graph ${\mathcal{G}}_0 = \{{\mathcal{V}}, {\mathcal{E}}, V, {\mathcal{Z}}\}$
with latent random noise vectors as initial latent edge representation.
The generators are permutation-equivarient functions.
They output node or edge features following the order of the input.

The critics receive alternatively real or generated graphs (without edge feature during the annotation step). The critics are permutation-invariant functions.
We provide details the generators and the critics in the next subsection.

The two steps (the two conditional GANs) are trained independently.
Using teacher forcing, we use real data for the conditioning during training.
For inference, we generate the edges features by conditioning on the previously generated node features.
The Figure \ref{fig:archi} illustrates the model architecture.

The software implementation of our method together with instructions for replicating our experiments are available at \url{https://github.com/yoboget/GrannGAN}.
In this section we provide some details of the implementation to help the reader understand important design decisions.

\begin{figure}[htp]
\centering
\includegraphics[width=0.8\textwidth]{archi.jpg}
\caption{GrannGAN architecture: grayscale boxes represent random noise, colored boxes represent node and edge features.}
\label{fig:archi}
\end{figure}

\subsection{Message passing neural network}\label{sec:MPNN}

In both the node- and the edge-annotation phases described in section \ref{sec:cGAN} all GrannGAN generator $g$ and critic $f$ functions are message passing neural networks (MPNN) \cite{gilmer_neural_2017}.
As mentioned here-above, the node-neighbourhood structure of the MPNNs is derived from the conditioning on the graph skeleton $\{{\mathcal{V}} {\mathcal{E}}\}$. Therefore, the skeleton acts on the $i$ and $j$ indices.

We use the following equations to perform $L$ update steps over the hidden states ${\mathbf{h}}_{i}^{(l)}$ and ${\mathbf{r}}_{ij}^{(l)}$ of each node and edge in the graph respectively
\begin{align}
{\mathbf{r}}_{ij}^{(l+1)} & = \phi_{r}^{(l)}([{\mathbf{h}}_{i}^{(l)}, {\mathbf{h}}_{j}^{(l)}, {\mathbf{r}}_{ij}^{(l)}]) \label{eq:edge} \\
\widetilde{{\mathbf{h}}}_{i}^{(l+1)} & = \sum_{j \in {\mathcal{N}}_i} \frac{1}{\sqrt{d_i}{\sqrt{d_j}}} {\mathbf{r}}_{ij}^{(l+1)} \label{eq:nodeaggreg} \\
{\mathbf{h}}_{i}^{(l+1)} & = \phi_h^{(l)}([{\mathbf{h}}_{i}^{(l)}, \widetilde{{\mathbf{h}}}_{i}^{(l+1)}]) \label{eq:node} \enspace .
\end{align}
Here $\phi$ are learned differentiable functions (we use small feedforward networks), ${\mathcal{N}}_i$ is the first order neighbourhood of node ${\mathbf{v}}_i$ as given by the graph skeleton ${\mathcal{S}}$, $d_i$ is its degree, and $[{\bm{a}},{\bm{b}}]$ is the concatenation of vectors ${\bm{a}}$ and ${\bm{b}}$.

We outline the whole generative pipeline in figure \ref{fig:archi}.
In the generator of the node-annotation phase $g_{\theta_{\mathcal{V}}}$ we initiate the node hidden states with a latent random noise vectors ${\mathbf{h}}_i^{(0)} = {\mathbf{z}}_i, \, \forall {\mathbf{v}}_i \in {\mathcal{V}}$ (and drop ${\mathbf{r}}_{ij}^{(0)}$ from inputs to $\phi_r^{(0)}$).
The generated node features are the hidden states of the last update step $\widehat{{\mathbf{v}}}_i = {\mathbf{h}}_{i}^{(L)}$.
In contrast, in the edge-annotation generator $g_{\theta_{\mathcal{E}}}$ the latent random noise is used for initiating the edge hidden states ${\mathbf{r}}_{ij}^{(0)} = {\mathbf{z}}_{ij}, \, \forall {\mathbf{e}}_{ij} \in {\mathcal{E}}$ while the node hidden states are the node features generated from the previous phase ${\mathbf{h}}_{i}^{(0)} = \widehat{{\mathbf{v}}}_i$.
The generated edge features are the hidden edge states of the last update step $\widehat{\mathbf{e}}_{ij} = {\mathbf{r}}_{ij}^{(L)}$.

The critic functions MPNNs are initiated from the real data examples as ${\mathbf{h}}_i^{(0)} = {\mathbf{v}}_i$ and ${\mathbf{r}}_{ij}^{(0)} = {\mathbf{e}}_{ij}$ or from the synthetic examples of the generators respectively.
To produce the critic scores, the last-step update functions ($\phi_h^{(L)}$ for the node critic and $\phi_r^{(L)}$ for the edge critic) have scalar outputs that are averaged to enter the loss in equation \eqref{eq:WGAN}. Therefore, the critic evaluates each node, aggregating information from all the nodes included in a radius of $2(L-1)$. While this receptive field may not cover the whole graph, we assume that the node and edge features can be evaluated locally.

It has been shown \cite{Arvind2020} that MPNN cannot capture graph substructures other than forests of stars. To alleviate this issue, we further embody the graph topology into the MPNNs by extending the node representation by a set of skeleton-related features. Similarly to \cite{Bouritsas2020}, we complement the node hidden states at the initial step ${\mathbf{h}}_i^{(0)}$ of all generators and critics by the node degree $d_i$ and the number of $k$-cycles (cycles of length $k$ the node is part of) extracted from the graph skeleton.

\subsection{Graph representations}\label{sec:representation}

An important property of graph representation ${\mathcal{G}}$ are their non-uniqueness.
There are in general $n!$ possible permutations $\pi$ determining the ordering of the nodes.
The particular choice of the representation ordering from the complete set $\Pi = \{ \pi_i \}_{i=1}^{n!}$ is therefore another source of stochasticity so that
\begin{align}\label{eq:permuations}
p({\mathcal{G}}) = p\big( \bigcup_{\pi \in \Pi} \pi({\mathcal{G}}) \big) = \sum_{\pi \in \Pi} p\big( \pi({\mathcal{G}}) \big) \enspace .
\end{align}

From equation \eqref{eq:permuations} we observe that when relying on the ordered graph representations ${\mathcal{G}}$, one shall in principle model the complete set of distributions $p(\pi({\mathcal{G}}))$ for all the $n!$ permutations $\pi$ to capture the unordered-set graph distribution $p({\mathcal{G}})$.
However, operating over the individual permutations $\pi({\mathcal{G}})$ would lead to inefficient use of the model capacity.
As an alternative, previous methods often fallback to operating over a single representation of each graph using some heuristic to fix the canonical ordering (such as the minimum depth-first-search \cite{goyal_graphgen_2020}, the unique breadth-first-search \cite{you_graphrnn_2018}  or the further sequentiallized domain driven SMILES \cite{daylight_chemical_information_systems_inc_daylight_2011}).
In result, the model has to learn not only the distribution of the graphs but also the ordering heuristic which is again inefficient from the perspective of generating the unordered graph sets ${\mathcal{G}}$.

A fundamental property of GrannGAN is that it is completely insensitive to the ordering $\pi({\mathcal{G}})$ that can in result be chosen arbitrarily.
This permuation equivariance\footnote{A function $f$ is equivariant with respect to permutation $\pi$ if $f(\pi({\bm{x}})) = \pi(f({\bm{x}})).$} of the message passing operations is pivotal for our method as it solves the problem of the graph representation non-uniqueness while allowing for efficient use of the model capacity.
\section{Related work}\label{sec:related}

GrannGAN is closely related to the broad category of graph generative models, an area which has attracted significant attention of the research community resulting in a flurry of papers in the last several years. A systematic review of the major advancements in the field can be found for example in the excellent survey \cite{faez_deep_2021}.

Unlike  previous work on that field, our method focuses on the particular problem of generating node and edge features conditionally over a given graph skeleton.
To the best of our knowledge, we are the first to investigate such structural-based conditional graph feature generation.

The closest to our settings are methods modelling the distribution of annotated graphs. These can be categorized into two large families: the ones generating graphs sequentially and those generating in the whole graph in one go.

\subsection{Sequential graph generation}
The first are methods generating the graphs sequentially, starting from small structures to which they gradually connect new graph components (nodes, edges, or complete sub-structures).
Most of these focus on generating chemical molecules and adopt various measures to improve the performance on this particular generative problem.
CharacterVAE \cite{gomez-bombarelli_automatic_2018} and GrammarVAE \cite{kusner_grammar_2017} rely directly on sequential SMILES representation of the molecules.
JT-VAE \cite{jin_junction_2018} operates over hand-crafted vocabulary of chemically valid sub-structures.
MolecularRNN \cite{popova_molecularrnn_2019} and GraphAF \cite{shi_graphaf_2019} generate the graphs by successive node and edge sampling steps
and ensure validity of the generated molecules by valency checking and the possibility for resampling at each of these steps.

Thanks to these domain-motivated components, the sequential methods often achieve excellent performance on molecular graphs datasets.
However, those directly relying on chemistry-specific data representations (e.g SMILES) cannot be easily extended to beyond the molecular problems.
On the other hand, methods not using the SMILES representation need to establish their own heuristic for the sequential traversal of the graph such as the breadth-first-search in GraphRNN \cite{you_graphrnn_2018} which in turn needs to be learned by the generative model together with the graph data distribution.
As any other sequential models, the graph autoregressive generators need to be particularly careful about capturing long-term dependencies, and suffer from slow sampling process.

Different from the previous models NetGAN \cite{netgan} uses GANs to learn the distribution of random walks over one big graph.

\subsection{Annotated graph generation in one go}
The other large family of models, into which GrannGAN can also be related, are those generating the graphs in one go, that is all the nodes and edges and their features together.
GraphVAE \cite{simonovsky_graphvae_2018} proposes to sidestep the problem of graph linearization characteristic for sequential models by relying on the variational auto-encoder framework \cite{DBLP:journals/corr/KingmaW13}.
Due to modelling the graph in the ordered representation of annotation and adjacency matrix, it needs to employ an expensive (though inexact) graph-matching procedure in the training loss calculation, which significantly hampers its scalability.
The flow-based graphNVP \cite{madhawa_graphnvp_2019} adopts the coupling strategy of \cite{DBLP:conf/iclr/DinhSB17} applying it to the rows of the graph annotation and adjacency matrices and thus preserving the ordering of the nodes and edges through the flow.
GraphNVP is the only model for annotated graph using permutation equivarient generative function.
In general, annotated graph generation in one go is still an open field of research.
Our contribution can be seen as a proposition to tackle this challenging issue.

Misc-GAN \cite{miscGan} aims to translate graphs from a source-domain graph into a target-domain graph using a multiscale cycle-GAN. It operates on unannotated graphs.

The closest to our GrannGAN is the MolGAN \cite{de_cao_molgan_2018} model, which uses GAN for molecule generation. Unlike in GrannGAN, the MolGAN generator is a feed-forward network sampling the ordered annotation and adjacency matrices of the graph representation. The authors of MolGAN observe that the model tends to suffer from mode collapse resulting in insufficient variation in the generated samples.  We suppose that this issue comes from the loss of capacity by learning to generate various permutations of the same graph and from an additional Reinforcement Learning module encouraging the generator to produce graphs with some specific properties. We never experienced mode collapse during our experiments. As we show it in the experiment section, our model presents excellent uniqueness and novelty rates.
We compare to MolGAN and other methods discussed here in our experiments in section \ref{sec:experiments}.

\section{Model details}\label{ann:model}

We describe here the details of our model architecture:

\subsection{General architecture}

The general architecture follows the design outlined in figure \ref{fig:method} in the main paper.
The GrannGAN is composed of two WGAN models, both following the same basic structure.

\subsubsection{Critics}

The critic functions $f$, in both the node-annotation and the edge-annotation models, are message passing networks (MPNNs) with 3 update steps.
The last step is complemented by node-wise and edge-wise MLP with the following architecture: Linear(64, 128), CELU, Linear(128, 128), CELU, Linear(128, 1).

\subsubsection{Generators}
The generator functions $g$, in both the node-annotation and the edge-annotation models, are MPNNs with 6 update steps.

\subsection{Message passing networks}

The basic architecture of all the MPNNs is described in equations \eqref{eq:edge}-\eqref{eq:nodeaggreg}.
In the QM9 model, the node update function $\phi_h$ is a sum of small MLPs operating indipendently over the previous hidden state and over the current node update.
In the ZINC model, we have introduced skip connections to both the edge \eqref{eq:edge} and node updates \eqref{eq:nodeaggreg}.

The update functions $\phi$ are all 2 layer, 64-dimensional MLPs with CELU activations.

The random noise variable vectors are ${\mathbf{z}} \in \mathbb{R}^{32}$ sampled from a Gaussian distribution ${\mathbf{z}} \sim {\mathcal{N}}(0, {\bm{I}})$.

We encode categorical edge and node features as one-hot vectors and use the Straight-Through Gumbel-Softmax approach \cite{jang_categorical_2016} for generating discrete samples with continuous approximation for backpropagation.
Continuous features are produced by a linear layer in the final step of the generator MPNN.

\section{Distribution comparisons}\label{ann:distribs}

We have further explored the bias that enforcing the chemical constraints bring into the distribution modelling.
In figure \ref{fig:graphAF} we compare the length measured in terms of the total number of atoms in the true underlying distribution and in the newly generated instances.
As the graphs illustrate, there is a large discrepency between these in the GraphAF model.
We hypothesise that by enforcing validity, the model strides away from the underlying distribution to parts of the space that may be easier from the chemical point of view.

Our model is restricted by the graph skeleton. When the skeletons are sampled directly from the underlying distribution, the distribution of the length of the generated graphs matches the true distribution perfectly by construction.

\begin{figure}[ht]
\centering
\includegraphics[width=0.4\textwidth]{pics/qm9_length}
\includegraphics[width=0.4\textwidth]{pics/zinc_length}
\caption{GraphAF: empirical distributions of the length (number of nodes) in the real and generated datasets}
\label{fig:graphAF}
\end{figure}