\section{Introduction}

Gradient-based optimization methods, such as SGD with momentum or Adam~\citep{kingma2014adam}, have become standard tools in the deep learning toolbox. While they are very effective for optimizing differentiable parameters, there is an interest in developing other efficient learning techniques that are complementary to gradient-based optimization. Evolution Strategies~\citep[ES,][]{Wierstra2014} is one such technique that has been used as an SGD alternative and has shown promising results on small-scale problems in reinforcement learning~\citep{igel2003neuroevolution, salimans2017evolution} and supervised learning~\citep{mandischer2002comparison, Lehman2017, Zhang2017, varelas2018comparative}.
ES is a black-box method and does not require the parameters to be differentiable. As such, it can potentially be applied to a much larger family of models than standard SGD.
Additionally, as ES only requires inference, it allows for training neural nets on inference-only hardware
(although we do not make use of this property here).
The goal of this paper is to explore how ES can be scaled up to larger and more complex models, from both an algorithmic and implementation perspective, and how it can be used in combination with SGD for training sparse neural networks.

We begin with investigating whether it is \emph{possible} to train CIFAR-10\xspace classification ConvNets with ES (\cref{sec:snes}).
This is a harder task than training MNIST classification MLPs \citep[as in][]{Zhang2017}, and to address it with ES we develop a more efficient execution model which we call semi-updates. As a result, ES reaches competitive accuracy compared to SGD on CIFAR-10\xspace, although this comes at the cost of significantly worse computational efficiency.
In~\cref{sec:hybrid} we then turn our attention to more practical applications (text-to-speech raw audio generation) and the use of ES alongside SGD for training large sparse models.
Instead of relying on hand-crafted methods for training such models by pruning weights~\citep{NarangDSE17, Zhu2017},
we employ ES to learn the weight sparsity masks (i.e.\ which weight should be used in the final sparse model), while SGD is responsible for learning the values of the weights, and is performed in parallel with ES.
It turns out that unlike the ES for weights, the ES for sparsity patterns needs significantly fewer parameter samples which makes this approach computationally feasible.
Beyond the scientific interest in the feasibility of such hybrid learning techniques, one practical advantage of this approach is that it enables joint training of weights together with their sparsity mask, thus avoiding the need to first train a dense model (which might be too large to fit into memory), and then prune it.
The experiments on the state-of-the-art sparse text-to-speech model show that ES achieves comparable performance to SGD with pruning.

In summary, our contributions are three-fold: (i) we propose a new execution model of ES which allows us to scale it up to more complex ConvNets on CIFAR-10\xspace;
(ii) we perform empirical analysis of the hyper-parameter selection for the ES training; (iii) we show how ES can be used in combination with SGD for hybrid training of sparse models, where the non-differentiable sparsity masks are learned by ES, and the differentiable weights by SGD.

\subsection*{Related work}
Prior works on hybrid SGD-ES methods mostly use ES for network structure and SGD to train the differentiable parameters of each sampled architecture \cite{yao1999evolving, igel2009genesis, real2018regularized}. This severely limits the size of the models where this method is practical as for each architecture sample the network has to be retrained.
Instead of meta-learning of the model, our goal is to optimize the model's non-differentiable and differentiable parameters \emph{jointly}.
Recent works compare the relation between the SGD gradients and ES updates, either on synthetic landscapes \cite{Lehman2017} or MNIST models \cite{Zhang2017}, while \cite{maheswaranathan2018guided} uses SGD gradients to guide the sampler's search direction. Instead of combining them, we use SGD and ES for two different training tasks - SGD for the continuous, differentiable parameters and ES for modelling the sparsity mask distribution.

Sparse models, where a subset of the model parameters are set to exactly zero, offer a faster execution and compact storage in sparse-matrix formats. This typically reduces the model performance but allows deployment to resource-constrained environments \cite{kalchbrenner2018efficient, Theis2018FasterGP}.
Training unstructured sparse supervised models can be traced back at least to \cite{Cun90optimalbrain}, which used second order information to prune weights.
More tractable efforts can be traced at least to \cite{Strom97sparseconnection}, which pruned weights based on the magnitude and then retrained.  That was more recently extended in \citet{HanLMPPHD16} where the pruning and retraining process was carried out in multiple rounds. The pruning process was further refined in \cite{NarangDSE17} where the pruning gradually occurred during the first pass of training so that it took no longer to find a pruned model than to train the original dense model.  Finally, the number of hyper-parameters involved in the procedure was reduced in \cite{Zhu2017}
and \citet{bellec2017deep} uses random walk for obtaining the sparsity mask. Additionally a number of Bayesian approaches have been proposed, such as Variational Dropout (VD) \cite{pmlr-v70-molchanov17a} or L0 regularization \cite{louizos2018learning}. VD uses reparametrization trick, while we learn directly the multinomial distribution with ES.
As indicated in \citet{gale19state}, VD and L0 regularization tends to perform not better than \cite{NarangDSE17} we compare against.

\section{Preliminaries}
\subsection{Natural Evolution Strategies}\label{ss:nes}

Black-box optimization methods are characterized by treating the objective function $f(\theta)$ as a black box, and thus do not require it to be differentiable with respect to its parameters $\theta$. Broadly speaking, they fall into three categories: population-based methods like genetic algorithms that maintain and adapt a set of parameters, distribution-based methods that maintain and adapt a distribution over the parameters $\theta \sim \pi$,
and Bayesian optimization methods that model the function $f$, using a Gaussian Process for example.

\emph{Natural evolution strategies} \citep[NES,][]{Wierstra2008,Wierstra2014} are an instance of the second type. NES proceeds by sampling a batch of $\esgst$ parameter vectors $\{\theta_i \sim \pi; 1 \leq i \leq \esgsm \}$, evaluating the objective for each ($f_i = f(\theta_i)$) and then updating the parameters of the distribution $\pi$ to maximize the objective $J = \mathbb{E}_{\theta \sim \pi}[f(\theta)]$, that is, the expectation of $f$ under $\pi$.
The attribute `natural' refers to the fact that NES follows the \emph{natural gradient} w.r.t. $J$ instead of the steepest (vanilla) gradient.

In its most common instantiation, NES is used to optimize unconstrained continuous parameters $\theta \in \mathbb{R}^d$, and employs a Gaussian distribution $\pi = \mathcal{N}(\mean, \mathrm{Cov})$ with mean $\mean \in \mathbb{R}^d$ and covariance matrix $\mathrm{Cov} \in \mathbb{R}^{d \times d}$.
For large $d$, estimating, updating and sampling from the full covariance matrix is too expensive, so here we use its diagonal approximation $\mathrm{Cov} = \operatorname{diag}(\stds^2)$ with element-wise variance terms $\stds^2 \in \mathbb{R}^d$; this variant is called Separable NES \citep[SNES,][]{snes}, and similar to \citep{ros2008simple}.
The sampled parameters are constructed as $\theta_i = \mean + \stds \odot \z_i$. where  $\odot$ is element-wise (Hadamard) product, from standard normal samples $\z \sim \mathcal{N}(0,\mathbf{I})$.
The natural gradient with respect to $(\mean, \stds)$ is given by:
\begin{eqnarray}\label{eq:snes}
\nabla_{\mean} J = \sum_{1 \leq i \leq n} u_i \z_i \notag, ~~
\nabla_{\stds} J = \stds \odot \sum_{1 \leq i \leq n} u_i \left(\z_i - 1 \right)^2
\end{eqnarray}
and the updates are
\begin{eqnarray}\label{eq:snes-update}
\mean \leftarrow \mean + \eta_{\mean} \stds \odot \nabla_{\mean} J & & \stds \leftarrow \stds \odot \exp(\frac{1}{2} \eta_{\stds} \nabla_{\stds} J )
\end{eqnarray}
where $\eta_{\mean}$ and $\eta_{\stds}$ are learning rates of the distribution's mean and variance parameters respectively, and $u_i$ is a transformation of the fitness $f_i$ (fitness-shaping, \cref{ss:fitness-shaping}).
Note that the multiplicative update on $\stds$ is guaranteed to preserve positive variances.

The more restricted case of constant variance ($\eta_{\stds}=0$), where the natural gradient coincides with the vanilla gradient, was advocated by~\citet{salimans2017evolution}, and it is sometimes referred to as simply `Evolution Strategies' (ES), even though it is subtly different from classic  Evolution Strategies~\citep{rechenberg1973evolutionsstrategie}: it matches the classic non-elitist $(1,\esgsm)$-ES only if the fitness shaping function is a top-$1$ threshold function.

\subsection{Fitness shaping functions}\label{ss:fitness-shaping}

It is often desirable to make evolution strategies more robust and scale-invariant by transforming the raw evaluation $f_i$ into a normalized utility $u_i$; this mapping is called \emph{fitness shaping}~\citep{hansen2001completely,Wierstra2014}.
A common one is based on the rank of $f_i$ within the current batch, where $\nu$ is a hyper-parameter:
\begin{equation}\label{eq:fshaping}
    u_i = \frac{\max\left(0, \log\left(\frac{\esgsm}{\nu} + 1\right) - \log(\operatorname{rank}(f_i))\right)}
    {\sum_{j=1}^\esgsm \max\left(0, \log\left(\frac{\esgsm}{\nu} + 1\right) - \log(j)\right)} - \frac{1}{\esgsm}
\end{equation}
For $\nu = 2$, this method sets a constant utility to the lowest $50\%$ of the fitnesses, which we use in our experiments.

\section{Scaling up SNES\xspace for supervised models}\label{sec:snes}

In this section we propose a novel method of distributing the evaluation of SNES\xspace updates. In \cref{s:semi-updates} we present the semi-updates method, which allows for better distribution of random number generation, fitness evaluation, and update computation among multiple workers. This allows us to train large supervised models with millions of parameters without creating a bottleneck on the master that holds the distribution parameters. Further on, in \cref{ss:es-experiments} we investigate various hyper-parameter selections for obtaining competitive results.

\subsection{Speeding up SNES\xspace with semi-updates}\label{s:semi-updates}
In practice, SNES\xspace training of a model with $d$ parameters requires a large amount of weight samples per generation (generation size $\esgsm$) that grows with the dimension of its parameters $\theta \in \mathbb{R}^{\esdimm}$.
A \emph{standard} execution model for SNES\xspace is to draw the parameter samples $\{\theta_{1 \dots \esgsm}\}$ at the master process, which looks after distribution parameters $(\mean, \stds)$, and to distribute them among $\esbsm$ worker processes. Even with smaller models, the main bottleneck is generating $\esdimm \cdot \esgsm$ random values which are needed for computing the weighted sums (\cref{eq:snes}).

One possible improvement is to send only the initial distribution parameters $(\mean, \stds)$ and a random seed to the \esbst workers to let each worker generate $\esgsm / \esbsm$ parameter samples and exchange set of $\esgsm / \esbsm$ fitness scalars, similar to \cite{salimans2017evolution}; we refer to this method as \emph{batched} execution. This significantly reduces the amount of data communicated between the workers, however to perform an update, a worker which performs the ES update still needs to generate \emph{all} \esgst random parameter samples, which is slow for large generations and models.

We propose to use \emph{semi-updates} execution model. It is similar to the batched method as each worker obtains only the distribution parameters $(\mean, \stds)$, however instead of sending back the fitness scalars for each sample, it computes the update on the batch of  $\esgsm / \esbsm$ parameter samples according to Equations~\ref{eq:snes} and~\ref{eq:snes-update}, and sends it back to the master.
Even though the standard ES execution model performs fitness shaping based on the rank of \emph{all} the parameter samples of a generation,
doing it on only $\esgsm / \esbsm$ parameter samples within each worker has a surprisingly little effect on the final performance. The main advantage of this method is that each worker now has to generate only $\esgsm / \esbsm$ parameter samples while the master  performs only a simple average over $\esbsm$ semi-updates. However, contrary to the batched method, the new distribution parameters has to be communicated to each worker.

\subsection{Experiments}\label{ss:es-experiments}
In this section we perform experiments with different variants of the SNES\xspace algorithm. In \cref{ss:semi-upd-exp}, we investigate the processing speed of different execution models defined in \cref{s:semi-updates}.
Later on, in \cref{ss:snes-cifar-exp} we investigate what is needed for being able to learn ConvNet models for CIFAR-10\xspace classification with SNES\xspace. Finally, in \cref{ss:es-nsamples}, we investigate the dependence of accuracy on the generation size.

We use SNES\xspace for supervised classification models and the objective function $f$ is the mini-batch log-likelihood\footnote{It is possible to use accuracy, however it under-performs due to quantization of the mini-batch accuracy.}.
For all experiments we optimize all model parameters $\theta \in \mathbb{R}^\esdimm$ with a single normal distribution. The distribution parameters are updated with learning rates $\eta_{\mean}=1$ and $\eta_{\stds}= (3 + \ln{\esdimm}) / (5 \sqrt{\esdimm})$ \citep[p.~33]{Wierstra2014}.
The mean $\mean$ is initialized using the truncated normal initializer \cite{glorot2010understanding}, and the variances $\stds^2$ are initialized to $1$.

We perform experiments on MNIST~\citep{lecun-98} and CIFAR-10\xspace~\citep{krizhevsky2009learning} datasets using \mbox{ConvNet} models, which are described in \cref{ss:models}. Selected models differ mainly in the number of parameters, which is controlled by the number of hidden representations. For CIFAR-10\xspace models, we train on random $24 \times 24$ crops and flips from the $32 \times 32$ training images. At test time, we process the full $32 \times 32$ test images and average-pool the activations before the classification layer. To reduce the parameter count, we use the Separable Convolution~\citep{chollet16xception}.

\subsection{Selected MNIST and CIFAR models}\label{ss:models}
In table \cref{tab:mnist_models} we provide details of the investigated MNIST and CIFAR models.
For the MNIST models, $\mathrm{C}_{\times n}^{M}$ stands for $M$ convolution filters of size $n \times n$, $\mathrm{P}$ for max pooling, and $\mathrm{F}_{a}^{b}$ for a fully connected layer with $b$ input features and $a$ output features. The MNIST-30k\xspace model is identical to the model by \citet{Zhang2017}.

Each CIFAR-10\xspace model consists of a set of $3\times3$ separable or standard convolutions where the number of filters per layer is specified in the column `Layers'. Layers with stride 2 are denoted as $s2$. The convolutional layers are followed by global average pooling and a fully connected layer with 10 outputs. The column `Sep' specifies whether separable or dense convolutions are used.

\begin{table}[h!]
    \centering
    \scriptsize
    \caption{MNIST models (left) and CIFAR-10\xspace models (right) used in the experiments. ACC @10k and @80k denotes the SGD performance on the test set after 10k or 80k steps of training respectively.
    }
    \label{tab:mnist_models}
    \setlength{\tabcolsep}{1pt}
    \begin{minipage}[t]{.45\linewidth}
        \begin{tabular}{l | r l | c } \toprule
        Name & \# Params & Layout & ACC @10k \\ \midrule
        MNIST-30k\xspace & $28\,938$ & \makecell{
            $\mathrm{C}_{\times5}^{16}$,
            $\mathrm{P}_{\times2}$,
            $\mathrm{C}_{\times2}^{32}$,\\
            $\mathrm{P}_{\times2}$,
            $\mathrm{F}_{10}^{1568}$}
        & $99.12\%$\\ \midrule

        MNIST-500k\xspace & $454\,922$ & \makecell{
            $\mathrm{C}_{\times5}^{32}$,
            $\mathrm{P}_{\times2}$,
            $\mathrm{C}_{\times2}^{64}$, \\
            $\mathrm{P}_{\times2}$,
            $\mathrm{F}_{128}^{3136}$,
            $\mathrm{F}_{10}^{128}$}
        & $99.28\%$\\ \midrule

        MNIST-3M\xspace & $3\,274\,634$ & \makecell{
            $\mathrm{C}_{\times5}^{32}$,
            $\mathrm{P}_{\times2}$,
            $\mathrm{C}_{\times2}^{64}$, \\
            $\mathrm{P}_{\times2}$,
            $\mathrm{F}_{1024}^{3136}$,
            $\mathrm{F}_{10}^{1024}$}
        & $99.33\%$\\
        \bottomrule
        \end{tabular}
    \end{minipage}
    \begin{minipage}[t]{.45\linewidth}
        \begin{tabular}{l  c c | c | r r } \toprule
        Name & Sep. & Layers & Params & \makecell{ACC\\@10k} & \makecell{ACC\\ @80k} \\ \midrule
        CIF-300k\xspace & YES
        & \makecell{
        $64$, $64$, \\
        $128_{s2}$, $128$, $128$,\\
        $256_{s2}$, $256$, $256$, \\
        $512_{s2}$}
        & $358\,629$
        & $87.7$ & $91.7$\\ \midrule

        CIF-900k\xspace  & YES
        & \makecell{
        CIF-300k\xspace \\
        + $512$, $512$ }
        & $895\,973$
        & $88.1$ & $92.01$\\ \midrule

        CIF-8M\xspace  & NO & \makecell{
        CIF-300k\xspace \\ + $512$, $512$}
        & $7\,790\,794$
        & $84.4$ & $94.6$\\

        \bottomrule
        \end{tabular}
    \end{minipage}
\end{table}

\subsubsection{Semi-updates}\label{ss:semi-upd-exp}
In \cref{fig:snes_scaling_emp}-left we compare the speed of different execution modes introduced in \cref{s:semi-updates}. The median time per generation is computed on a MNIST-500k\xspace model on 110 workers where each worker has up to 10 CPU cores available, thus for each generation size $\esgsm$, the amount of computational resources is constant. As it can be seen, the semi-updates provide a significant speedup.

\begin{figure}
    \centering
    \setlength{\tabcolsep}{1pt}
    \begin{minipage}[t][][t]{.33\linewidth}
        \includegraphics[width=1\linewidth]{./figures/speed_uber_mnist_log.pdf}
    \end{minipage}%
    \begin{minipage}[t][][t]{.33\linewidth}
        \includegraphics[width=1\linewidth]{figures/snes_tinysep_fixvarb.pdf}
    \end{minipage}%
    \begin{minipage}[t][][t]{.33\linewidth}
        \includegraphics[width=1\linewidth]{figures/snes_tinysep_bn.pdf}
    \end{minipage}

    \caption{
    Left - Seconds per generation of the MNIST-500k\xspace model (lower is better)
    for different execution modes, trained on 110 workers with varying generation sizes $\esgsm$ (x-axis in log scale). Batches and semi-updates are computed over $\esbsm = \esgsm / 100$.
    The proposed semi-updates scheme achieves the highest training speed and allows for larger generation sizes.
    Center - Performance of SNES\xspace on CIF-300k\xspace for different training data batch regimes. VarB computes the fitness on a variable batch per each SNES sample; FixB computes the fitness on a single batch fixed for the entire SNES generation; WFixB fixes the training batch only within each worker, and different workers use different batches.
    Right - Performance of SNES\xspace on CIF-300k\xspace with and without Batch Normalisation.
    }

    \label{fig:snes_scaling_emp}
\end{figure}

\subsection{Training on CIFAR-10\xspace with SNES\xspace}\label{ss:snes-cifar-exp}

We have found that using SNES\xspace for training models on CIFAR-10\xspace dataset requires a careful selection of hyper-parameters. To achieve 99\% test set accuracy on MNIST with SNES\xspace, it is typically sufficient to use a large number of parameter samples \esgst. But for CIFAR-10\xspace,
it turned out to be challenging to achieve performance comparable to SGD.

Due to the computational complexity, we perform most of the hyper-parameter selection experiments on a relatively small model with approx. 300k parameters -- CIF-300k\xspace, and train it for 10k steps. It uses  a parameter-efficient architecture based on separable convolutions~\citep{chollet16xception}.
This model is bigger than MNIST-30k\xspace, but smaller than MNIST-3M\xspace.
Each experiment is run on 50 Tesla P4 GPU workers and the average processing time is 20s per generation.
For all the experiments, the results are computed over a generation size $20\,000$. Each generation is by default evaluated on a batch of 256 training images.

\paragraph{Batch normalization}
Somewhat surprisingly, batch normalization (BN) \citep{ioffe2015batch}, is crucial for obtaining competitive results with SNES\xspace. BN is often seen as an architectural block to simplify training with SGD, but it turns out that it is even more important for SNES\xspace training, as shown in \cref{fig:snes_scaling_emp}-right. Similar effect has also been observed for RL ES models \citep{salimans2017evolution, Mania2018}.

\paragraph{Fixed versus variable batch}
For MNIST experiments, using a different batch of training examples in each sample of a generation provides better results than using the same fixed batch, as first noted in \cite{Zhang2017}. However, this does not hold for the CIFAR-10\xspace experiments where we have consistently observed that it is important to evaluate each SNES\xspace sample within a generation on the same training batch, as shown in \cref{fig:snes_scaling_emp}-middle. We hypothesize that this could be due to the higher complexity of CIFAR-10\xspace classification task compared to MNIST, which leads to increased variance of the fitness values when using different batches.

With semi-updates, fitness is computed individually by each worker, so it is important to fix the training data only within each worker for each semi-update. In fact, fixed batch per semi-update, WFixB, obtains slightly better performance than FixB (\cref{fig:snes_scaling_emp}-middle) due to more training data per generation. It also allows for a simpler implementation as the training data does not have to be synchronized among the workers.

\subsection{Convergence and generation size}\label{ss:es-nsamples}
Finally, we test the SNES\xspace performance versus the number of parameter samples \esgst per generation. In all experiments which we have performed with SNES\xspace, this has proven to be the most important parameter for improving performance, as observed in \cite{Zhang2017}.
For the MNIST models, we have run the training for 10k steps. Results are shown in \cref{tab:snes_nsamples}-left. For the MNIST-30k\xspace model, it is possible to achieve a slightly better accuracy of $99\%$ at 10k steps vs $98.7\%$ in \cite{Zhang2017}.
For the MNIST-3M\xspace we are able to achieve higher performance ($98.81\%$ test set acc.) than \cite{Zhang2017} mainly due to larger number of training steps, which was facilitated by the more efficient semi-updates execution model.

\begin{table}[h]
    \centering
    \caption{Test set accuracy after 10k and 20k training steps for MNIST (left) and CIFAR-10\xspace (right) dataset respectively with SGD and SNES\xspace with various generation sizes \esgst.}
    \label{tab:snes_nsamples}
    \footnotesize
    \setlength{\tabcolsep}{2pt}

    \begin{tabular}{c c c}
        \begin{tabular}{c  c  c c c c c}\toprule
             & SGD &\multicolumn{5}{ c }{Generation size \esgst}  \\
             Model & Acc & 1k & 5k & 10k & 20k & 50k  \\ \midrule
             MNIST-30k\xspace & 99.04
             & 98.57 & 98.76 & 99.13 & 99.18 & 99.16 \\
             MNIST-500k\xspace & 99.28
             & 97.36 & 98.87 & 98.84 & 99.19 & 99.08 \\
             MNIST-3M\xspace & 99.33
             & 96.54 & 98.46 & 98.74 & 98.60 & 98.81 \\
             \bottomrule
        \end{tabular}
        & ~ &
        \begin{tabular}{c  c   c c c c}\toprule
             & SGD &\multicolumn{4}{ c }{Generation size \esgst}  \\
             Model & Acc & 10k & 50k & 100k & 200k \\ \midrule
             CIF-300k\xspace & 88.7
             & 80.03 & 86.32 & 88.17 & 89.48 \\
             CIF-900k\xspace & 88.79
             & 81.48 & 87.08 & 88.47 & 89.08 \\
             \bottomrule
        \end{tabular}
    \end{tabular}
\end{table}

In \cref{tab:snes_nsamples}-right, we show the accuracy obtained with SNES\xspace for the CIFAR-10\xspace models. The models use batch normalization, fixed mini-batch of 256 training images per worker (WFixB).  Similarly to the MNIST models, it is possible to reach performance comparable to SGD with a sufficient number of parameter samples.

\paragraph{Number of training samples per evaluation}
Similarly to SGD \cite{Smith2017}, SNES\xspace performance can be improved by evaluating fitness function on more training examples (batch size), as can be seen in \cref{tab:snes_batchsize}.
We hypothesize that this is due to reduced variance of the fitness values. However, in our experiments generation size $\esgsm$ tended to have a larger effect on the final performance.

\begin{table}[h]
    \footnotesize
    \caption{Performance of SNES\xspace on CIF-300k\xspace versus the size of the training mini-batch.}
    \label{tab:snes_batchsize}
    \centering
    \begin{tabular}{c| c c c} \toprule
         Batch Size & 128 & 256 & 512 \\ \midrule
         Val Acc & $78.61\%$ & $81.12\%$ & $83.03\%$ \\
         Gen [s] & 13.31 &  20.05 & 33.3 \\ \bottomrule
    \end{tabular}
\end{table}

\subsection{Discussion}

The empirical results show that with the right algorithmic and engineering changes, it is possible to scale up SNES\xspace and tackle tasks of higher complexity. We believe that with sufficient effort SNES\xspace can be scaled to even larger models than we have shown. Compared to standard SGD, SNES\xspace only needs to infer the model, which has a potential to enable training neural nets on inference-only or inference-optimized hardware.

\section{Hybrid ES\xspace for sparse supervised models}\label{sec:hybrid}

\begin{algorithm}[h] \footnotesize
 \KwIn{fitness $f$, diff. params. $\theta_{init}$, sparsity control $k$, temperature $\tau$, learning rates $\eta_{\bm{l}}, \eta_{\theta}$, steps number $S$}
 $\bm{l} \leftarrow 0$ \;
 \For{Step = 1 \dots $S$}{
    \For{i = 1 \dots n}{
      sample mask $\mathbf{m}_i \sim \mathcal{C}(\mathbf{p})$ (sampled $k$ times)\;
      $\theta_i \leftarrow \mathbf{m}_i \odot \theta$\;
      evaluate the fitness $f(\theta_i)$;~compute $\nabla_{\theta_i} f$\;
    }
    compute utilities $u_i$\;
    compute $\nabla_{\theta} f = \frac{1}{n}\sum_{i=1}^n  \nabla_{\theta_i} f$\;
    update differentiable weights $\theta = \theta + \eta_{\theta} \nabla_{\theta} f$\;
    compute $\nabla_{\bm{l}} J = \frac{1}{\tau} \sum_{i=1}^{n} u_i \cdot (1 - \mathbf{p}) \odot \mathbf{m}_i$\;
    update mask distribution $\bm{l} \leftarrow \bm{l} +  \eta_{\bm{l}} \nabla_{\bm{l}} J$\;
 }
 \caption{C-ES\xspace, a hybrid of ES and SGD}\label{alg:sgd_es}
\end{algorithm}

We have shown that it is possible to use ES\xspace for learning differentiable parameters of large supervised models. However, a key advantage of black-box optimization algorithms is that they are able to train models with non-differentiable parameters.
In this section we show that it is indeed possible to combine conventional SGD optimization with ES\xspace for learning weight sparsity masks of sparse neural networks. We use SGD to train the differentiable weights, and ES\xspace for learning the masks at the same time. This leads to \emph{C-ES\xspace}, a hybrid ES\xspace-SGD scheme which works as follows.
At each training step, a generation of mask samples is drawn from the sparsity mask distribution. Each mask sample zeroes out a subset of model weights, and the resulting sparse weights are then evaluated to obtain the mask fitness, used to compute the mask update with ES\xspace.
Simultaneously, each worker performs gradient descent w.r.t.\ its current non-zero weights, and the weight gradients are then averaged across all workers to perform a step of SGD with momentum.
The algorithm is specified in \cref{alg:sgd_es}.

This method, similarly to Drop-Connect \citep{wan2013regularization}, randomly zeroes model parameters. However, we replace the constant uniform distribution with a distribution optimized by ES.

\subsection{Sparsity mask distributions}
Mask samples $\mathbf{m}_i \in \{0,1\}^d, i = 1 \dots n$, are modelled with a multinomial distribution.
To sample a mask, we draw repeatedly $\catnsm$ indices from a categorical distribution $\mathcal{C}(\mathbf{p})$, where $\catnsm$ controls the number of non-masked weights.
We model the distribution probabilities $\mathbf{p}$ with a softmax function $p_j = \operatorname{exp}(\lval{j}/\tau) / \Sigma_c \operatorname{exp}(\lval{c}/\tau)$ where $\tau$ is temperature and $\bm{l} \in \mathbb{R}^\esdimm$ is a vector of distribution parameters, learned with ES\xspace.
For each sample $j \sim \mathcal{C}(\mathbf{p})$, we set $m_j \leftarrow 1$, i.e.\@\xspace we sample which model parameters are retained (not zeroed out), and the model is evaluated with $f(\mathbf{m \odot \theta})$.

We approximate the derivatives of the multinomial distribution with derivatives of the $\mathcal{C}$'s PMF $g(x = j \vert \mathbf{p}) = p_j$:
\begin{equation}
    \frac{\partial \ln g(x \vert \mathbf{p})}{\partial l_j} =
    \begin{cases}
             1 - p_j &\mbox{if } x=j    \\
            0 &\mbox{if } x\neq j
            \end{cases}
\end{equation}
and an ES update from $\esgsm$ mask samples is approximated as:
\begin{equation}
\nabla_{\bm{l}} J \leftarrow \sum_{i=1}^{n} u_i \cdot \frac{1 - \mathbf{p}}{\tau} \odot \mathbf{m}_i
\end{equation} where $u_i$ is the utility of sample $i$. We do not use natural gradients.
The sparsity mask distribution parameters are updated with $\bm{l} \leftarrow \bm{l} + \eta_{\bm{l}} \cdot \nabla_{\bm{l}} J$ with learning rate $\eta_{\bm{l}}$.

An alternative way of modelling weight sparsity is using separate Bernoulli distribution for each differentiable parameter \citep{Williams1992}.
However, it does not allow for controlling the overall sparsity of the network, as the sparsity of each weight is learned independently from the others.
Although it might be possible to control the sparsity of Bernoulli-parameterized masks using a special form of regularization as in~\citep{louizos2018learning}, in this work we opted for modelling the sparsity masks with a multinomial distribution as described above, as it allows for a direct control over the sparsity. Details about sampling without replacement are in the following section.

\subsection{Sampling from multinomial distributions}\label{ss:cat-samplers}
A mask $\mathbf{m}$, sampled from a multinomial distribution, is an outcome of $\catnsm$ draws from a categorical distribution $\mathcal{C}$.
We have observed that the implementation of the mask sampler is crucial. Not only it does affect the ability to scale to large models, but it also has a significant influence on the final performance.

The standard multinomial sampler, referred to as MS-wR\xspace, implements sampling from categorical distribution with replacement. For this method, as the number of sampler invocations $\catnsm$ increases, fewer unique indices are sampled due to the increased number of collisions, i.e.\@\xspace $\| \mathbf{m} \|_1 < \catnsm$.

Fixed sparsity can achieved with various methods.
As a baseline which achieves $\| \mathbf{m} \|_1 = \catnsm$, we sample  $\catnsm - \| \mathbf{m} \|_1$ additional unique non-zero indices, uniformly from the non-masked ones. This method is referred to as MS-wR+u\xspace. This method does not sample exactly from the original distribution but gives uniform mass to the remaining samples.

However, in our experiments sampling exactly $\catnsm$ indices without replacement tends to yield better results. Unfortunately, we have not found any computationally efficient GPU implementation of this, so instead we consider two different approximations.
The first, MS-woRb\xspace, splits $\catnsm$ into $m$ batches. For each batch we sample $\catnsm/m$ indices using MS-wR\xspace and remove them from the distribution. This method has the advantage that for $m = \catnsm$, it converges to exact sampling without replacement, while for small $m$ it is more computationally efficient. However, unless $m = \catnsm$, it does not guarantee $\| \mathbf{m} \|_1 = \catnsm$.

In the other approximation, MS-tN\xspace, we sample $M \times \catnsm$ indices from the categorical distribution (where $M>1$ is a sufficiently large number), accumulate the sampled indices in a histogram and use its top-$\catnsm$ indices to form the mask. To break possible ties, we add small uniform noise $\mathcal{U}(0, 10^{-3})$ to the histogram\footnote{Stable sort, used in many top-N implementations, is by definition biased towards lower indices.}.

\paragraph{Fast sampler implementations}
An efficient implementation of the random sampler for $\mathcal{C}(\mathbf{p})$ is crucial for the execution speed.
The current multinomial sampler in Tensorflow is based on Gumbel-softmax sampling \citep{maddison2017concrete, jang2017gumbel}.
The downsides are that it requires generating $k \esdimm$ random values and the same amount of storage.  When $\esdimm$ is in the order of millions and $k$ itself is a tenth of that size (for $90\%$ sparse models), the product $\frac{\esdimm^2}{10}$ is prohibitively large, both in terms of the computation cost and memory.
A more efficient strategy is to use inverse CDF sampling. It computes the CDF of the distribution using a prefix sum \cite{blelloch1990pre, harris2007parallel} in $O(\esdimm)$ time, then generates $k$ random numbers and performs a sorted binary search into the CDF in $O(k \log \esdimm)$ with storage requirements only $O(k + \esdimm)$.
We employ CUB\footnote{\url{https://nvlabs.github.io/cub/}} to implement fast prefix sums, and use existing fast GPU RNG generators to implement a fast GPU binary search similar to the one in Thrust~\cite{bell2012thrust}.

\subsection{Experimental results}
In this section we evaluate C-ES\xspace for training sparse feed-forward and recurrent models.
First, we deploy the proposed method on feed-forward CIFAR-10\xspace classification models CIF-300k\xspace and CIF-8M\xspace (see \cref{ss:models}), and sparsify all layers apart from the last, fully connected layer.
We then show that C-ES\xspace is also applicable to a different kind of models,
and use it to train SparseWaveRNN\xspace~\cite{kalchbrenner2018efficient}, which is a state-of-the-art recurrent text-to-speech model.

\subsubsection{Feed-forward models}\label{ss:exps-ff}
In this section we show results for the feed-forward image classification models.
First we investigate hyper-parameter selection, and then we compare the results against the pruning baseline \cite{NarangDSE17} which progressively masks model parameters with magnitude close to zero.
By default, in all the following experiments with C-ES\xspace training, the models are trained from the start with an initial sparsity $50\%$. The ES learning rate is set to $\eta_{\bm{l}} = 0.1$ and softmax temperature is set to $\tau = 3$ and $\esgsm=9$. We use a single C-ES\xspace distribution for all weights. At test time, the mask is formed by the top-$\catnsm$ indices from $\bm{l}$.

\paragraph{Sampling methods}

In \cref{tab:cates_samplers} we show results for different sampling methods introduced in \cref{ss:cat-samplers}. This experiment is performed on the CIF-300k\xspace for $k = \esdimm / 2$ ($50\%$ sparsity), which is trained for 10k training steps. As can be seen, MS-wR\xspace, which does not ensure constant sparsity, reaches the worst performance. With MS-wR+u\xspace, where we sample additional indices from a uniform distribution, the final accuracy is considerably improved. The variants of MS-woRb\xspace increase the accuracy even further, and the best performing method is MS-tN\xspace. We believe this is due to the fact that this method amplifies higher probabilities, and keeps a constant number of non-zero weights while still sampling model parameters with lower mask probabilities from the distribution.

\begin{table}[h]
    \centering
    \footnotesize
    \caption{Performance versus sampling method after 10k training steps, $\esgsm=200$. Dense test accuracy $87.65\%$.}
    \label{tab:cates_samplers}
    \setlength{\tabcolsep}{3pt}
    \begin{tabular}{c| c c  | c c c | c c c} \toprule
         Test Acc & \multirow{2}{*}{wR} & \multirow{2}{*}{wR+u}
         & \multicolumn{3}{c|}{woRb, m=} & \multicolumn{3}{c}{tN, M=} \\
         $[\%]$ & & & $3$ & $4$ & $5$ & $2$ & $3$ & $5$ \\ \midrule
         CIF-300k\xspace & $61.8$ & $74.2$ & $83.5$ & $83.4$ & $83.7$ & $85.6$ & $87.6$ & $88.6$
         \\ \bottomrule
    \end{tabular}
\end{table}

In all experiments of the main text, we use MS-tN\xspace approximation with $M=5$.

\paragraph{C-ES\xspace and generation size}
Empirically, we have observed that the generation size has a surprisingly limited effect on C-ES\xspace training, as can be seen in \cref{tab:cates_cifsep_gs}. As ConvNets are often trained with DropOut \cite{srivastava14a} or its close variant DropConnect \cite{wan2013regularization} as a form of regularization, they are known to be robust against randomly dropped weights or activations. We hypothesize that because of this robustness, finding a sparsity mask distribution for a current local optima of the differentiable weights, towards which the SGD converges, is a considerably simpler task.
Conveniently, small $\esgsm$ allows us to make use of standard pipelines for multi-GPU SGD training.

\begin{table}[h]
    \centering
    \footnotesize
    \caption{Test set accuracy after 20k training steps of C-ES\xspace for 50\% sparsity with different generation size \esgst.
    }
    \label{tab:cates_cifsep_gs}
    \setlength{\tabcolsep}{4pt}
    \begin{tabular}{c | c c c c c c} \toprule
        Test Acc & \multicolumn{6}{c}{Generation Size \esgst} \\
         $[\%]$ & 2 & 5 & 10 & 50 & 100 & 200  \\ \midrule
        CIF-300k\xspace & 87.33 & 88.13 & 88.48 & 88.54 & 88.32 & 88.72
        \\ \bottomrule
    \end{tabular}
\end{table}

\paragraph{Comparison to pruning}
In this section we compare C-ES\xspace against a more conventional pruning-based method for training sparse models~\cite{NarangDSE17}.
For C-ES\xspace, the update of the distribution is computed over 9 parameter samples per generation.
We use a similar sparsity schedule for both algorithms: the models are trained for $2000$ steps with the initial sparsity ($0\%$ for pruning and $50\%$ for C-ES\xspace), and then the sparsity is decreased monotonically until it reaches the final sparsity at step 50k. Overall, training is performed for 80k steps, and we use SGD with momentum, weight decay, and the learning rate follows a schedule\footnote{$0.1$ for 40k steps, then $0.01$, and $0.001$ for the last 20k steps.}.

The results of the C-ES\xspace and pruning methods and the execution speed are summarized in \cref{tab:cifar_cates}-left.
Generally, C-ES\xspace provides competitive results to the pruning baseline.
As C-ES\xspace allows us to train sparse models from the first training step, we test different initial sparsities in \cref{tab:cifar_cates}-right, which shows that C-ES\xspace is remarkably stable across initial sparsities even for models trained with only $10\%$ dense weights from scratch. In this experiment we additionally compare against a model trained with a fixed mask distribution -- ``FixMask''-- where we set the ES learning rate to zero. It shows that training of sparse over-parameterised models, such as CIF-8M\xspace is possible even with a fixed sparsity mask, however it fails for models with fewer parameters where learning the mask is important.

\begin{table}[h]
    \centering
    \caption{Accuracy on the test set of the C-ES\xspace trained on CIFAR-10\xspace with constant initial sparsity $50\%$ (Left) and a constant target sparsity of $90\%$ (right). Results after 80k steps. FPS for the C-ES\xspace method measured on 3 NVIDIA Tesla P100 GPU.
    }
    \label{tab:cifar_cates}
    \footnotesize
    \setlength{\tabcolsep}{2pt}
    \begin{minipage}[t]{.49\linewidth}
        \begin{tabular}{l l  c c c c  c} \toprule
            &   & \multicolumn{4}{c }{Final sparsity} &   \\
            Model & Method  & $0\%$ & $50\%$ & $80\%$ & $90\%$ & FPS \\ \midrule
            \multirow{2}{*}{CIF-300k\xspace} & Pruning
            & \multirow{2}{*}{91.73}
            & 90.98 & 88.02 & 80.62
            & 30.6\\
            & C-ES\xspace
            &
            & 89.13 & 84.98 & 79.58
            & 3.4\\
            \midrule

            \multirow{2}{*}{CIF-900k\xspace} & Pruning
            & \multirow{2}{*}{92.01}
            & 90.67 & 88.37 & 81.74
            & 25.8\\
            & C-ES\xspace
            &
            & 89.01 & 86.1 & 81.94
            & 2.6\\
            \midrule

            \multirow{2}{*}{CIF-8M\xspace} & Pruning
            & \multirow{2}{*}{94.6}
            & 93.32  & 92.64 & 92.24
            & 20.7\\
            & C-ES\xspace
            &
            & 93.94 & 92.87
            & 90.83
            & 1.32\\
            \bottomrule
        \end{tabular}
    \end{minipage}~~%
    \begin{minipage}[t]{.49\linewidth}
        \begin{tabular}{ c  c  c c c c c} \toprule
            & & \multicolumn{5}{ c }{Initial sparsity} \\
             Model & Method & $10\%$ & $30\%$ & $50\%$ & $70\%$ & $90\%$ \\ \midrule
             \multirow{2}{*}{CIF-300k\xspace}
             & C-ES\xspace &  80.15 & 79.46 & 79.86 & 80.23 & 79.58 \\
             & FixMask\xspace & 69.84 & 64.22 & 71.28 & 71.24 & 65.79 \\
             \midrule

            \multirow{2}{*}{CIF-900k\xspace}
             & C-ES\xspace & 80.87 & 81.73 & 81.94 & 82.61 & 83.76\\
             & FixMask\xspace & 69.69 & 72.91 & 74.78 & 74.67 & 74.74 \\
             \midrule

             \multirow{2}{*}{CIF-8M\xspace}
             & C-ES\xspace & 90.21 & 90.16 & 90.56 & 90.96 & 91.77 \\
             & FixMask\xspace & 90.01 & 90.01 & 90.37 & 90.59 & 90.46 \\
             \bottomrule
        \end{tabular}
    \end{minipage}
\end{table}

\subsubsection{Recurrent models}
In this section we show the results on a large sparse recurrent network, SparseWaveRNN\xspace~\cite{kalchbrenner2018efficient} for text-to-speech task.
We trained it on a dataset of 44 hours of North American English speech recorded by a professional speaker.
The generation is conditioned on conventional linguistic features and pitch information. All compared models synthesize raw audio at 24 kHz in 16-bit format. The evaluation is carried out on a held-out test set.
We perform experiments on two models -- one with 448 (WR448\xspace) and another with 1792 hidden state variables (WR1792\xspace). As in~\cite{kalchbrenner2018efficient}, we do not sparsify the 1D convolutions at the network input, which has approximately $8M$ parameters. In total, the WR448\xspace has $420k$ and WR1792\xspace $23.7M$ masked parameters.

For all experiments with the C-ES\xspace training, models are trained with an initial sparsity $50\%$ and generation size $\esgsm = 8$ on 8 GPUs.
The sparsity decreases after 40k steps and reaches the final sparsity after 251k steps. Model parameters are trained with ADAM optimizer and a constant learning rate $2 \cdot 10^{-4}$. Otherwise, we use the same C-ES\xspace hyper-parameters as in  \cref{ss:exps-ff}.
We use a separate mask distribution per each parameter tensor which offers slightly better execution speeds.
As noted in~\cite{kalchbrenner2018efficient}, in practical applications it might be beneficial to have the sparsity pattern in the form of contiguous blocks, so
we train the models with different sparsity block widths (width $1$ corresponds to unconstrained sparsity pattern as in the experiments above). This is implemented by using the same sparsity mask value for several contiguous weights.

\begin{table}[h]
    \centering
    \caption{NLL on the test set at 300k steps (lower is better) of the recurrent Sparse WaveRNN models. Left - Comparison of pruning and C-ES\xspace on various models and block widths. Processing speed (FPS) is measured on 8 NVIDIA Tesla P100 GPU, for block width 1. Right - C-ES\xspace versus FixMask trained on WR1792\xspace for different initial sparsities, block width, and target sparsity of $97\%$.}
    \label{tab:res_wavernn}
    \footnotesize
    \setlength{\tabcolsep}{2pt}

    \begin{minipage}[t]{.49\linewidth}
        \begin{tabular}{l l | c | c | c c  c | c } \toprule
            \multirow{2}{*}{Model} & \multirow{2}{*}{Method} & SGD & \multirow{2}{*}{Spar.} &  \multicolumn{3}{c |}{Block Width}  & \multirow{2}{*}{FPS}   \\
             & & NLL &   &  1 & 2 & 16 &  \\\midrule
            \multirow{2}{*}{WR448\xspace} & Pruning & \multirow{2}{*}{5.72} & \multirow{2}{*}{$50\%$}
            & 5.81 & 5.78  & 5.75
            & 1.16 \\
            & C-ES\xspace & &
            & 6.02 & 5.91 & 5.80
            & 1.06  \\
            \midrule
            \multirow{2}{*}{WR1792\xspace} & Pruning & \multirow{2}{*}{5.43} & \multirow{2}{*}{$97\%$}
            & 5.49 & 5.48 & 5.52
            & 0.49 \\
            & C-ES\xspace & &
            & 5.64 & 5.61 & 5.56
            & 0.45  \\
            \bottomrule
        \end{tabular}
    \end{minipage}%
    \begin{minipage}[t]{.49\linewidth}
        \centering

        \setlength{\tabcolsep}{2pt}
        \begin{tabular}{ c  l  | c c c c} \toprule
            & & \multicolumn{3}{ c }{Initial sparsity} \\
             Model & Method & $20\%$ & $50\%$ & $80\%$ & $90\%$ \\ \midrule
             \multirow{2}{*}{WR1792\xspace BW1}
             & C-ES\xspace & 5.66 & 5.64 & 5.63 & 5.63 \\
             & FixMask\xspace & 5.73 & 5.72 & 9.54 & 9.54 \\ \midrule
             \multirow{2}{*}{WR1792\xspace BW16}
             & C-ES\xspace & 5.58 & 5.56 & 5.56 & 5.57  \\
             & FixMask\xspace & 5.65 & 5.65 & 9.54 & 9.54 \\
             \bottomrule
        \end{tabular}
    \end{minipage}%

\end{table}

The results are shown in \cref{tab:res_wavernn}.
The computational overhead of C-ES\xspace is approximately equal to the generation size, however C-ES\xspace is easily parallelizable over multiple computational devices.
With each mask sample being evaluated on a single GPU, the execution speed is comparable to pruning, even though more than 57M random numbers have to be generated per training step on each GPU.
In all cases, the C-ES\xspace method is competitive with the pruning baseline. In general, it performs better for larger block widths due to the reduced number of C-ES\xspace parameters.
However, contrary to pruning, C-ES\xspace allows us to train the sparse model from scratch -- as shown in \cref{tab:res_wavernn}-right, which opens up possibilities for using accelerated sparse training, even though we have not investigated this further here. Contrary to feed-forward models, a fixed mask  distribution (``FixMask'') does not work well for high initial sparsities.
Additionally, C-ES\xspace might allow for optimizing such non-differentiable factors as execution speed.

\section{Conclusion}

In this work we have investigated the applicability of Evolution Strategies to training more complex supervised models. We have shown that using appropriate ``tricks of the trade'' it is possible to train such models to the accuracy comparable to SGD.
Additionally, we have shown that hybrid ES is a viable alternative for training sparsity masks, allowing for training sparse models from scratch in the same time as the dense models when the ES samples are parallelized across multiple computation devices.
Considering that ES is often seen as a prohibitively slow method only applicable to small problems, the significance of our results is that ES should be seriously considered as a complementary tool in the DL practitioner's toolbox, which could be useful for training non-differentiable parameters (sparsity masks and beyond) in combination with SGD. We hope that our results, albeit not the state of the art, will further reinvigorate the interest in ES and black-box methods in general.

\paragraph{Acknowledgements} We would like to thank David Choi and Jakub Sygnowski for their help developing the infrastructure used by this work.


\section{Introduction}
\label{intro}
The so - called non - dissipative transport effects have been widely discussed recently \cite{Landsteiner:2012kd,semimetal_effects7,Gorbar:2015wya,Miransky:2015ava,Valgushev:2015pjn,Buividovich:2015ara,Buividovich:2014dha,Buividovich:2013hza}. In the high energy physics those effects are expected to be observed in the non - central heavy ion collisions, when the fireballs are in the presence of both magnetic field and rotation~\cite{ref:HIC}. Such effects have also been considered for the recently discovered Dirac and Weyl semimetals \cite{semimetal_effects6,semimetal_effects10,semimetal_effects11,semimetal_effects12,semimetal_effects13,Zyuzin:2012tv,tewary,16}.
It was expected that their family includes, in particular,  the chiral separation effect (CSE) \cite{Metl}, the chiral magnetic effect (CME)  \cite{CME,Kharzeev:2015znc,Kharzeev:2009mf,Kharzeev:2013ffa}, the chiral vortical effect (CVE) \cite{Vilenkin}, the anomalous quantum Hall effect (AQHE)  \cite{TTKN,Hall3DTI,Zyuzin:2012tv}.

It is widely believed that all mentioned above phenomena have the same origin - the chiral anomaly. For example, the direct "derivation" of the CME from the standard expression for chiral anomaly has been presented \cite{Zyuzin:2012tv,CME,SonYamamoto2012}.
Nevertheless, the more accurate consideration demonstrates that the original equilibrium version of the CME does not exist. In \cite{Valgushev:2015pjn,Buividovich:2015ara,Buividovich:2014dha,Buividovich:2013hza} this was demonstrated using the numerical lattice methods. In the context of condensed matter theory the absence of CME was reported for the particular model of Weyl semimetal \cite{nogo}. The same conclusion has been obtained using the conjectured no - go Bloch theorem \cite{nogo2}. The analytical proof of the absence of the equilibrium CME was presented in \cite{Z2016_1,Z2016_2} using the technique of Wigner transformation \cite{Wigner,star,Weyl,berezin}  applied to the lattice models\footnote{The CME may, though, survive in a certain form as a non - equilibrium kinetic phenomenon - see, for example,  \cite{ZrTe5}.}. The same technique allowed to reproduce the known results on the AQHE \cite{Z2016_1} and to confirm the existence of the equilibrium CSE \cite{KZ2017} both in the framework of the quantum field theory and the solid state physics.

Below we review the mentioned above technique, which allows to express through the topological invariants in momentum space the response of various nondissipative currents to the external magnetic or electric field.
In the framework of the naive nonregularized quantum field theory the CSE was discussed, for example, in \cite{Gorbar:2015wya} in the technique similar to the one that was used for the consideration of the CME \cite{CME} in Dirac semimetals and AQHE in Weyl semimetals \cite{Zyuzin:2012tv}.  However, the more refined calculations reported here confirm the existence of the CSE and AQHE in the mentioned systems and demonstrate the absence of the equilibrium CME \cite{Z2016_1,Z2016_2,KZ2017}.

It is worth mentioning that the investigation of momentum space topology was developed previously mainly within the condensed matter physics theory. It allows to relate the gapless boundary fermions to the bulk topology for the topological insulators and to describe the stability of the Fermi surfaces and the Fermi points, as well as the fermion zero modes on vortices (for the review see \cite{Volovik2003,Volovik:2011kg}). It is worth mentioning that momentum space topology of QCD has been discussed recently in \cite{Z2016_3}. The whole Standard Model of fundamental interaction has been considered as a topological material in \cite{Volovik:2016mre}.

\section{Lattice models in momentum space}

\label{SectWigner}

Here we consider briefly the lattice models in momentum space following the methodology of \cite{Z2016_1,Z2016_2}.
In the absence of the external gauge field  the partition function of the theory defined on the infinite lattice is
\begin{equation}
Z = \int D\bar{\psi}D\psi \, {\rm exp}\Big( - \int_{\cal M} \frac{d^D {p}}{|{\cal M}|} \bar{\psi}^T({p}){\cal G}^{-1}({ p})\psi({p}) \Big)\label{Z1}
\end{equation}
 Here $|{\cal M}|$ is the volume of momentum space $\cal M$, $D$ is the dimensionality of space - time. $\bar{\psi}$ and $\psi$ are the Grassmann - valued fields defined in momentum space $\cal M$. The Green function $\cal G$ is specific for the given system. For the often used model of $3+1$ D Wilson fermions it is given by
 \begin{equation}
 {\cal G}({p}) = - i \Big(\sum_{k}\gamma^{k} g_{k}({p}) - i m({p})\Big)^{-1}\label{G10}
 \end{equation}
 where $\gamma^k$ are Euclidean Dirac matrices
while
$g_k({p})$ and $m({p})$ are the real - valued functions ($k = 1,2,3,4$) given by
\begin{equation}
g_k({p}) = {\rm sin}\,
p_k, \quad m({p}) = m^{(0)} +
\sum_{a=1,2,3,4} (1 - {\rm cos}\, p_a)\label{gWilson}
\end{equation}
The fields in coordinate space are related to the fields in momentum space:
$\psi({r}) = \int_{\cal M} \frac{d^D {p}}{|{\cal M}|} e^{i {p}{r}} \psi({p})$.
 This allows to define formally the values of fields at any other values of $r$, not only at the lattice sites. The partition function has the form
\begin{equation}
Z = \int D\bar{\psi}D\psi \, {\rm exp}\Big( - \sum_{{r}_n} \bar{\psi}^T({r}_n)\Big[{\cal G}^{-1}(-i\partial_{r})\psi({ r})\Big]_{{r}={r}_n} \Big)\label{Z2}
\end{equation}
Here the sum in the exponent is over the discrete coordinates ${r}_n$. However, the operator $-i\partial_{r}$ acts on the function $\psi({r})$ of continuous variable defined above.

\section{Introduction of the gauge field}

In the presence of the constant external gauge field corresponding to the potential $A(x)$ (up to the terms irrelevant in the low energy effective theory) we may represent the partition function as follows \cite{Z2016_1,Z2016_2}
\begin{eqnarray}
Z = \int D\bar{\psi}D\psi \, {\rm exp}\Big( -  \int_{\cal M} \frac{d^D {p}}{|{\cal M}|} \bar{\psi}^T({p})\hat{\cal Q}(i{\partial}_{p},{p})\psi({p}) \Big)\label{Z4}
\end{eqnarray}
Here
\begin{equation}
\hat{\cal Q} = {\cal G}^{-1}({p} - {A}(i{\partial}^{}_{p}))\label{calQM}
\end{equation}
while the pseudo - differential operator ${A}(i\partial_{p})$ is defined as follows. First, we represent the original gauge field ${A}({r})$ as a series in powers of coordinates ${r}$. Next, variable ${r}$ is substituted in this expansion by the operator $i\partial_{p}$. Besides, in Eq. (\ref{calQM}) each product of the components of ${p} - {A}(i{\partial}^{}_{p})$ is subsitituted by the symmetric combination (for the details see \cite{Z2016_1}).

Electric current is the response of effective action to the variation of the external Electromagnetic field. This gives
\begin{eqnarray}
j^k({R}) &=& \int_{\cal M} \frac{d^D {p}}{(2\pi)^D} \,  {\rm Tr} \, \tilde{G}({R},{p}) \frac{\partial}{\partial p_k}\Big[\tilde{G}^{(0)}({R},{p})\Big]^{-1}\label{j423}
\end{eqnarray}
where the Wigner transformation of the Green function is expressed as:
\begin{equation}
 \tilde{G}({R},{p}) = \sum_{{r}={r}_n} e^{-i {p} {r}} G({R}+{r}/2,{R}-{r}/2)\label{Wl2}
\end{equation}
while the Green function itself is
\begin{eqnarray}
G({r}_1,{r}_2)&=& \frac{1}{Z}\int D\bar{\Psi}D\Psi \,\bar{\Psi}({r}_2)\Psi({r}_1) {\rm exp}\Big(-\sum_{{ r}_n}\Big[ \bar{\Psi}({r}_n)\Big[{\cal G}^{-1}(-i\partial_{r}  \nonumber\\&& - {A}({r}))\Psi({r})\Big]_{{r}={r}_n}\Big]\Big)\nonumber
\end{eqnarray}
At the same time $
\tilde G^{(0)}({R},{p})  = {\cal G}({p}-{A}({R}))\label{Q0}
$.
In \cite{Z2016_1} the following expression was derived for the linear response of the electric current to external electromagnetic field: \begin{eqnarray}
j^{(1)k}({R})  &= & \frac{1}{4\pi^2}\epsilon^{ijkl} {\cal M}_{l} A_{ij} ({R}), \label{calM}\\
{\cal M}_l &=& \int_{} \,{\rm Tr}\, \nu_{l} \,d^4p \label{Ml} \\ \nu_{l} & = &  - \frac{i}{3!\,8\pi^2}\,\epsilon_{ijkl}\, \Big[  {\cal G} \frac{\partial {\cal G}^{-1}}{\partial p_i} \frac{\partial  {\cal G}}{\partial p_j} \frac{\partial  {\cal G}^{-1}}{\partial p_k} \Big]  \label{nuG}
\end{eqnarray}

Here tensor $\cal M$ is the topological invariant in momentum space, i.e. it is not changed if the system is modified smoothly. This representation allows to prove that the equilibrium CME does not exist for the system of massless fermions. The proof is presented in \cite{Z2016_2}. At the nonzero chiral chemical potential  $\mu_5$ we may consider the following expression for the fermion Green function in the absence of the gauge field:
 \begin{equation}
 {\cal G}^{}({\bf p}) = \Big(\sum_{k}\gamma^{k} g_{k}({\bf p}) + i\gamma^4 \gamma^5 \mu_5 - i m({\bf p})\Big)^{-1}\label{G2}
 \end{equation}
 where $g_k({\bf p})$ and $m({\bf p})$ are the real - valued functions, $k = 1,2,3,4$. Function $m$ is defined in such a way that the pole of $\cal G$ appears that corresponds to the massless fermion.
We may substitute ${\cal G}$ of Eq. (\ref{G2}) into Eq. (\ref{calM}) while dealing with the linear response to the external magnetic field. In the non - marginal cases including the ordinary regularization of the QFT using Wilson fermions we are able to bring the system using smooth transformation from the state with nonzero $\mu_5$ and vanishing fermion mass to the state with vanishing $\mu_5$ and nonzero fermion mass. During such a modification the singularity of the Green function is not encountered. Therefore, the value of the topological invariant ${\cal M}_4$ responsible for the CME is not changed. It may be calculated easily for the system of massive fermions, and this calculation gives ${\cal M}_4 =0$, which proves the absence of the corresponding CME current.

\section{AQHE in the $3+1$D systems}
\label{SectHall3d}

From the above expressions it follows that the Hall current is given by \cite{Z2016_1}
\begin{equation}
{j}^k_{Hall} = \frac{1}{4\pi^2}\,{\cal M}^\prime_l\,\epsilon^{jkl}E_j,\label{HALLj3d}
\end{equation}
where ${\cal M}^\prime_l = i{\cal M}_l/2$ is
\begin{eqnarray}
{\cal M}^\prime_l &=&  \frac{1}{3!\,4\pi^2}\,\epsilon_{ijkl}\,\int_{} \,\,d^4p\,{\rm Tr} \Big[  {\cal G} \frac{\partial {\cal G}^{-1}}{\partial p_i} \frac{\partial  {\cal G}}{\partial p_j} \frac{\partial  {\cal G}^{-1}}{\partial p_k} \Big]  \label{nuGHall}
\end{eqnarray}
Notice, that in the case of the non - interacting condensed matter system with the Green function  ${\cal G}^{-1} = i \omega - \hat{H}({\bf p})$ expressed through the Hamiltonian $\hat{H}$ we may derive (similar to the above considered case of the $2+1$ D system) the representation of the components of topological invariant ${\cal M}^{\prime}_l$ with $l\ne 4$ through the Berry curvature $ {\cal F}_{ij}$:
\begin{eqnarray}
{\cal M}^\prime_l &=&  \frac{\epsilon^{ijl}}{4\pi}\sum_{\rm occupied}\, \int d^3p\, {\cal F}_{ij}
\end{eqnarray}
Here the sum is over the occupied branches of spectrum.

The formalism developed in \cite{Z2016_1} allows to demonstrate that for the wide class of Weyl semimetals the contribution to the AQHE electric current of the pair of Weyl fermions is given by
\begin{equation}
{j}^k_{Hall} = \frac{\beta}{2\pi^2}\,\epsilon^{jk3}E_j,\label{HALLj3dp}
\end{equation}
Here $E$ is electric field while $\beta$ is the distance in momentum space between the Weyl fermions of opposite chirality while the third axis is directed along the line connecting them. Thus the previously obtained result on the AQHE in Weyl semimetals is confirmed. Moreover, the same method allows to predict the existence of the topological insulators with the AQHE, in which the meaning of the constant $\beta$ is the length of the inverse lattice vector (proportional to $1/a$, where $a$ is the lattice spacing).

\section{Linear response of axial current to external magnetic field}

In continuum theory the naive expression for the axial current is $\langle \bar{\psi} \gamma^\mu \gamma^5 \psi\rangle$. Several different definitions for the particular lattice regularization may give this expression in the naive continuum limit. In \cite{KZ2017} it has been proposed to define the axial current in the lattice models as follows
\begin{eqnarray}
j^{5k}({R}) &=& \int_{\cal M} \frac{d^D {p}}{(2\pi)^D} \,  {\rm Tr} \,\gamma^5\,  \tilde{G}({R},{p}) \frac{\partial}{\partial p_k}\Big[\tilde{G}^{(0)}({R},{p})\Big]^{-1}\label{j423}
\end{eqnarray}
where
\begin{eqnarray}
&&\tilde G^{(0)}({R},{p})  = {\cal G}({p}-{A}({R}))\label{Q01}
\end{eqnarray}
One can easily check that in the naive continuum limit this definition gives $\langle \bar{\psi} \gamma^k \gamma^5 \psi\rangle$.

Let us regularize the expressions for the massless fermions using the finite temperature version of the lattice theory. For the periodic boundary conditions in the spatial directions
and anti-periodic in the imaginary time direction, the lattice momenta are
\begin{equation}\label{disc}
p_i \in (0, 2\pi);\, p_4=\frac{2\pi}{N_t }(n_4+1/2)
\end{equation}
where $i=1,2,3$ while $n_4=0,...,N_t-1$. Temperature is equal to $T = 1/N_t $, in lattice units $1/a$, where $a$ is the lattice spacing. Thus the imaginary frequencies are discrete $p_4 = \omega_{n}=2\pi T (n+1/2)$, where $n= 0, 1, ... N_t-1$, while the axial current is expressed via the Green's functions as follows:
\begin{eqnarray}
j^{5k}&=&-\frac{i}{2}T\sum_{n=0}^{N_t-1}\int \frac{d^3p}{(2\pi)^3}{\rm Tr}\, \gamma^5 ({\cal G}(\omega_{n},\textbf{p})\partial_{p_{i}}{\cal G}^{-1}(\omega_{n},\textbf{p})\nonumber \\&&\partial_{p_{j}}{\cal G}(\omega_{n},\textbf{p})\partial_{p_{k}}{\cal G}^{-1}(\omega_{n},\textbf{p}))F_{ij}\label{calN}
\end{eqnarray}

Next, we introduce the chemical potential in the standard way $\omega_{n} \to \omega_{n}-i\mu$. If $\gamma^5$ anti - commutes with the Green function in a small vicinity of its poles, then the term in the axial current linear in  $\mu$ is \cite{KZ2017}
\begin{equation}
j^{5k}= \frac{{\cal N}\,\epsilon^{ijk}}{4\pi^2} F_{ij} \mu\label{jmuH}
\end{equation}
in the low temperature limit $T\to 0$. Here
\begin{eqnarray}
{\cal N}&=&\frac{\epsilon_{ijk}}{12}\int_{\Sigma}\frac{d^3p}{(2\pi)^2}{\rm Tr}\, \gamma^5 {\cal G}(\omega_{},\textbf{p})\partial_{p_{i}}{\cal G}^{-1}(\omega_{},\textbf{p})\nonumber \\&&\partial_{p_{j}}{\cal G}(\omega_{},\textbf{p}) \partial_{p_{k}}{\cal G}^{-1}(\omega_{},\textbf{p})\label{calN1}
\end{eqnarray}
while $\Sigma$ is the 3D hypersurface of infinitely small volume that embraces the singularities of the Green function concentrated at the Fermi surfaces (or Fermi points). The advantage of this representation is that Eq. (\ref{calN}) is the topological invariant.

In particular, in \cite{KZ2017} it has been demonstrated using the regularization with Wilson and overlap fermions that if the lattice model describes one massless Dirac fermion, then
\begin{equation}
{\cal N}= 1
\end{equation}
In this way the validity of the obtained earlier naive result for the CSE has been established.

In the presence of nonzero mass $m^{(0)}$ the situation is changed. At $\mu \ge m^{(0)}$ the Fermi surface appears, and it contributes to the chiral current through Eq. (\ref{calN}). However, the result is not given by the simple expression of Eq. (\ref{jmuH}).

\section{Conclusions}
\label{sec-1}

Above we reviewed the application of momentum space topology to the analysis of anomalous transport. This methodology works equally well both in the lattice regularized relativistic quantum field theory and for the lattice models of solid state physics. In particular, it appears that the response of electric current to the external electric field is expressed through the topological invariant in momentum space. Its nonzero value leads to the appearance of the quantum Hall effect. This allows to calculate the AQHE conductivity for the wide class of the condensed matter systems. The same technique allows to prove the absence of the equilibrium CME in the lattice regularized quantum field theory: the corresponding conductivity is proportional to the topological invariant in momentum space that vanishes for the systems with finite chiral chemical potential. For the systems of massless fermions the further development of this technique allows to express through the topological invariant the terms in the axial current proportional to the external magnetic field and to the ordinary chemical potential. This is the topological invariant in momentum space responsible for the stability of the Fermi point. In this way we describe the axial current of the CSE that is relevant for description of the fireballs that appear in the non - central heavy ion collisions.

We show that in the considered cases the corresponding nondissipative currents are proportional to the momentum space topological invariants. They are not changed when the system is deformed smoothly, without passing through a phase transition. The corresponding conductivities for the  complicated systems may be calculated within the simple ones related to the original systems by a smooth deformation. On the technical side our methodology is based on the derivative expansion applied to the Wigner transform of the two - point Green functions. We introduce the slowly varying external gauge field directly to the momentum space formulation of the lattice models. The external gauge field appears as a pseudo - differential operator ${\bf A}(i\partial_{\bf p})$. This way of the incorporation of the external field to the theory is  useful for the analytical derivations and allows to obtain the above mentioned remarkable relation between the momentum space topological invariants and the non - dissipative currents.

We started our consideration from Eq. (\ref{Z1}) thus neglecting the contributions of interactions. However, even the complicated interacting system may be described in a certain approximation by the same Eq. (\ref{Z1}). Then function $\cal G$ incorporates all contributions of interactions to the two - point fermion Green functions. In this approach we neglect the contributions to the physical observables of the fermion Green functions with more than two external lines.

MZ kindly acknowledges numerous discussions with G.E.Volovik. Both authors are greatful to M.N.Chernodub for useful discussions. The work of ZK was supported by Russian Science Foundation Grant No 16-12-10059.


\section{Introduction}

In recent years, propagation of the superparticle [1] and of the superstring
[2] in the presence of external background superfields was intensively
investigated [3--7]. It was shown in the original paper [3] that the
requirement of the Siegel's local fermionic symmetry [8] in the $d=10$
superparticle action (correct inclusion of interaction has to preserve
local symmetries of a theory) led to some constraints on a background.
Namely, the full set of SYM constraints arised in the case of SYM
background superfield and some part of the full set of supergravity
constraints appeared in the case of a curved background. Different
attempts to get the full set of supergravity constraints proceeding from the
consideration of the superparticle in a curved background were proposed
in Refs. [4, 9].

The straightforward way to get the superparticle model in a curved
background includes three steps [3]. First of all, the flat action and
gauge transformations are to be written in terms of the flat
supervielbein. Second, the coordinates and supervielbein are set to be
the coordinates and supervielbein of a curved superspace. Third, since
the resulting action doesn't possess any local symmetry except the
reparametrization invariance, it is necessary to demand the
$k$-transfor\-mations (arising at the first and second steps) to be a
symmetry of the superparticle action. One can note, however, that this
procedure doesn't guarantee that the arising in such a way gauge
transformations are of the most general form. In the present paper we
consider the (1,0) superparticle in (1,0) curved superspace.
Proceeding from the minimal set of constraints which must be
imposed on background geometry for correct inclusion of
interaction, we find the most general form of Siegel's local fermionic
transformations for the theory. An algebraic structure of the
arising transformations is investigated and requirements leading
to the full set of (1,0) supergravity constraints are presented.

We begin with brief consideration of the model in flat superspace in
Sec. 2. A full gauge algebra of the theory is shown to be closed
off-shell, as opposed to the case of another dimensions. In Sec. 3
we examine the model in curved superspace within the Hamiltonian
formalism and find the minimal set of constraints on background
necessary for correct inclusion of interaction. We do these using
the Dirac's constraint formalism and requiring a correct number
of first- and second-class constraints for the model. Our analysis here
is closely related to that of Ref. [4]. Next, in the Lagrangian
formalism we reconstruct the most general form of Siegel's
transformations which are consistent with these restrictions on the
background. It is interesting to note that the arising transformations
contain non-trivial contributions including the torsion superfield and
don't coincide with the direct generalization of the flat Siegel's
transformations. In Sec. 4 full gauge algebra is evaluated and shown to
be closed off-shell and nontrivially deformed as compared to the flat
one. Further, we construct the formulation leading to the full set of
(1,0) supergravity constraints. The full set follows from
requirement that the direct generalization of the flat gauge
transformations is realized in the model. One can note the similarity
with the (1,0) chiral superparticle [10]. In the conclusion the
possibilities of extending the results to the case of another dimensions
are discussed.

\section{Off-shell closure of local symmetries}

We use the real representation for $\Gamma^M$-matrices in $d=2$ and
light-cone coordinates for bosonic variables. (1,0) superspace is
parametrized by the coordinates $z^M=(x^+,x^-,\theta^\alpha )$, where
$\theta^\alpha \equiv \left(\begin{array}{l}\theta^{(+)}\\
0 \end{array} \right)$ is a Majorana--Weyl spinor. In these notations
the action for the (1,0) superparticle in flat superspace has the
following form:
$$
S = \int {\rm d}\tau \,e^{-1}\Pi^+{\dot x}^-
\eqno{(1)}$$
where we denoted $\Pi^+={\dot x}^+-{\rm i}\dot\theta^{(+)}
\theta^{(+)}$ and $e$ is an auxiliary einbein field. The theory possesses
global (1,0)-supersymmetry
$$
\delta\theta^{(+)}=\epsilon^{(+)}, \qquad \delta x^+ = {\rm i}
\theta^{(+)}\epsilon^{(+)}.
\eqno{(2)}$$
The local symmetries of the model include the standard
reparametrization invariance with a bosonic parameter $\alpha (\tau )$.
$$
\begin{array}{l} \delta_\alpha x^+ =\alpha{\dot x}^+, \qquad
\delta_\alpha x^- =\alpha{\dot x}^-,\\
\delta_\alpha \theta^+ =\alpha{\dot\theta}^+, \qquad \delta_\alpha e
=(\alpha e)^{\cdot}.\end{array}
\eqno{(3)}$$
and the Siegel's transformations with a fermionic parameter $k^{(-)}
(\tau )$
$$
\begin{array}{l}\delta_k\theta^{(+)}=\Pi^+k^{(-)}, \qquad
\delta_kx^-=0,\\
\delta_kx^+={\rm i}\delta_k\theta^{(+)}\theta^{(+)}, \qquad \delta_ke =
2{\rm i}ek^{(-)}\dot\theta^{(+)}.\end{array}
\eqno{(4)}$$
An interesting property of the superparticle in (1,0) flat superspace
is off-shell closure of gauge algebra for the model (it is not so for
another dimensions). To check this assertion, consider the following
transformations with a bosonic parameter $\xi^-$
$$
\delta_\xi x^-=-\xi^-\Pi^+{\dot x}^-, \qquad \delta_\xi e = \xi^-e^2
(e^{-1}\Pi^+)^{\cdot}.
\eqno{(5)}$$
It is a trivial symmetry of the action in the sense that it vanishes
on-shell and, consequently, doesn't remove a number of degrees of
freedom of the theory. In the presence of this symmetry, however, the
full gauge algebra turns out to be closed and has the following form:
$$
\begin{array}{ll}{} &\qquad\alpha = 2{\rm i}k_2^{(-)}k_1^{(-)}\Pi^+\cr
[\delta_{k_1}\delta_{k_2}] = \delta_\alpha + \delta_\xi + \delta_{k_3},
&\qquad\xi^-=2{\rm i}k_2^{(-)}k_1^{(-)}\\
{}&\qquad k_3^{(-)}=2{\rm i}k_2^{(-)}k_1^{(-)}\dot\theta^{(+)};\cr
[\delta_k\delta_\xi ]=\delta_{\xi_1}, &\qquad
\xi_1^-=2{\rm i}\xi^-k^{(-)}\dot\theta^{(+)}\cr
[\delta_{\xi_1},\delta_{\xi_2}]=\delta_{\xi_3}; &\qquad
\xi_3^-=(\xi_1^-\dot\xi_2^--\xi_2^-\dot\xi_1^-)\Pi^+;\cr
[\delta_k,\delta_\alpha ]=\delta_{k_1}, &\qquad k_1^{(-)} = \alpha{\dot
k}^{(-)}-\dot\alpha k^{(-)};\cr
[\delta_\xi ,\delta_\alpha ]=\delta_{\xi_1}, &\qquad \xi_1^-=\alpha
\dot\xi^--2\xi^-\dot\alpha ;\cr
[\delta_{\alpha_1},\delta_{\alpha_2}]=\delta_{\alpha_3}, &\qquad
\alpha_3=\alpha_2\dot\alpha_1-\alpha_1\dot\alpha_2.
\end{array}
\eqno{(6)}$$
An application of the Dirac procedure to the model (1) leads to the
following constraints system
$$
P_e\approx 0, \qquad P_{(+)}+{\rm i}\theta^{(+)}P_+\approx 0, \qquad
P_+P_- \approx 0
\eqno{(7)}$$
where ($P_e,P_+,P_-,P_{(+)}$) are canonically conjugate momenta for the
variables ($e,x^+,x^-,\theta^{(+)}$) respectively. Specific feature of
$d=2$ lies in the fact that the last condition implies two
possibilities:\\
a) $P_+=0,~P_-\neq 0$ and, consequently, all constraints are first
class;\\
b) $P_-=0,~P_+\neq 0$ whence we conclude that $P_e\approx 0$ and $P_-
\approx 0$ are first class while $P_{(+)}+{\rm i}\theta^{(+)}P_+
\approx 0$ is second class.

\section{The model in a curved background}

Introducing a set of basis one-forms $e^A={\rm d}z^M{e_M}^A$, where
${e_M}^A$ is the supervielbein, the extension of the (1,0) superparticle
action to curved superspace can be written in the form
$$
S=\int{\rm d}\tau\,e^{-1}\,{\dot z}^M{e_M}^+{\dot z}^N{e_N}^-
\eqno{(8)}$$
where $M=(m,(+))$ are coordinate indices and $A=(+,-,(+))$ are tangent
space indices of (1,0) curved superspace. World indices appear on the
coordinates $z^M=(x^m,\theta^{(+)})$ and supervielbein only.

It should be noted that in the presence of an arbitrary background
superfield the action (8) doesn't possess any local symmetry except
the reparametrization invariance ($\delta_\alpha e =(\alpha
e)^{\cdot}$, $\delta_\alpha z^M=\alpha{\dot z}^M$). Thus it is not the
superparticle model yet. Let us find what constraints on background
geometry follow from the requirement of correct number of degrees of
freedom for the theory or, what is the same, from the requirement of
correct number of first and second class constraints for the model.

The momenta conjugate to $e$ and $z^M$ are
$$
P_e = 0, \qquad P_M=e^{-1}{e_M}^+{\dot z}^N{e_N}^- +e^{-1}{e_M}^-{\dot
z}^N{e_N}^+
\eqno{(9)}$$
whence one gets the primary constraints
$$
P_e\approx 0, \qquad P_{(+)}\approx 0
\eqno{(10)}$$
where we denoted $P_A\equiv {e_A}^MP_M$.

The canonical Hamiltonian is given by
$$
H=P_e\lambda_e+eP_+P_- + \lambda^{(+)}P_{(+)}
\eqno{(11)}$$
where $\lambda_e$, $\lambda^{(+)}$ are Lagrange multipliers enforcing
the constraints (10). The graded Poisson bracket has the form
$$
\{AB\}=(-)^{\epsilon_A\epsilon_N}\frac{\vec\partial A}{\partial z^N}~
\frac{\vec\partial B}{\partial P^N}-(-)^{\epsilon_A\epsilon_B +
\epsilon_B\epsilon_N}\frac{\vec\partial B}{\partial z^N}~
\frac{\vec\partial A}{\partial P^N}
\eqno{(12)}$$
where $\epsilon_A$ is a parity of a function $A$. It is straightforward
to check that
$$
\{P_A,P_B\}={T_{AB}}^CP_C-\left({\omega_{AB}}^C-(-)^{\epsilon_A
\epsilon_B}{\omega_{BA}}^C\right)P_C
\eqno{(13)}$$
where ${T_{AB}}^C$ are components of the torsion two-form
$$
{T_{BC}}^A=(-)^{\epsilon_B(\epsilon_M+\epsilon_C)}{e_C}^M{e_B}^N\left(
\partial_N{e_M}^A+(-)^{\epsilon_N(\epsilon_M+\epsilon_B)}{e_M}^B
{\omega_{NB}}^A -\right.
$$
$$
\qquad{}\left.-(-)^{\epsilon_N\epsilon_M}(\partial_M{e_N}^A +
(-)^{\epsilon_M(\epsilon_N+\epsilon_B)}{e_N}^B{\omega_{MB}}^A)\right)
\eqno{(14)}$$
and ${\omega_{MA}}^B$ are components of the superconnection one-form
which, in the case of $d=2$, can be chosen in the form [12]
$$
{\omega_{NA}}^B=\omega_N{\delta_A}^BN_B, \qquad N_B=(N_+,N_-,N_{(+)}) =
(1,-1,1/2).
\eqno{(15)}$$
The preservation in time of the primary constraint $P_e\approx 0$ leads
to the secondary one
$$
P_+P_-\approx 0.
\eqno{(16)}$$
As in flat case, this condition implies two possibilities which should
be considered separately.

Because inclusion of interaction has to preserve dynamical contents of
a theory (a number of degrees of freedom), in the first case we must
demand all constraints to be first class. It means that all brackets
between the constraints have to vanish weakly. Taking into account Eq.
(15) and the fact that ${T_{AB}}^C=-(-)^{\epsilon_A\epsilon_B}
{T_{BA}}^C$, it is easy to get the following weak equations
$$
\begin{array}{c}
\{P_+,P_+\}=0,\qquad\{P_+,P_{(+)}\}\approx T_{+(+)}{}^-P_-, \\
\{P_{(+)},P_{(+)}\}\approx T_{(+)(+)}{}^-P_-\end{array}
\eqno{(17)}$$
whence we conclude that it is necessary to require the following
constraints on background
$$
{T_{+(+)}}^-=0, \qquad {T_{(+)(+)}}^-=0.
\eqno{(18)}$$
In the second case, the constraints $P_e\approx 0$, $P_-\approx 0$ are
to be first class while the constraint $P_{(+)}\approx 0$ is to be
second class. Evaluating brackets between the constraints
$$
\begin{array}{c}
\{P_-,P_-\}=0,\qquad\{P_-,P_{(+)}\}\approx T_{-(+)}{}^+P_+, \\
\{P_{(+)},P_{(+)}\}\approx T_{(+)(+)}{}^+P_+\end{array}
\eqno{(19)}$$
we conclude that they have correct class if the following conditions:
$$
{T_{-(+)}}^+=0, \qquad {T_{(+)(+)}}^+=-2{\rm i}\Phi
\eqno{(20)}$$
are fulfilled, where $\Phi$ is a nonvanishing everywhere arbitrary
superfield (the multiplier -2i is taken for further convenience).

Thus, a correct formulation for the (1,0) superparticle model in a
curved background implies Eqs. (18) and (20).

Now, let us reconstruct local transformations which are consistent
with Eqs. (18), (20) in the Lagrangian formalism. The following remark
will be important here. Since ${e_M}^A(z)$ and $\omega_A(z)$ are background
superfields, they transform under arbitrary variation $\delta z^M$ only
through their coordinate dependence ($\delta{e_M}^A=\delta z^N\partial_N
{e_M}^A$). Thus, local Lorentz and gauge transformations are not
consistent. Following the standard arguments [12], we accompany $\delta
z^N$ by compensating local Lorentz transformation $\delta {e_M}^A=
\delta z^N\partial_N{e_M}^A+\delta_L{e_M}^A$ with the $\delta
z^N$-dependent parameter $L$ such that the resulting $\delta{e_M}^A$ is
Lorentz vector. The results are
$$
\begin{array}{l}\delta{e_M}^A=\delta z^N{\rm D}_N{e_M}^A\equiv \delta
z^N\left(\partial_N{e_M}^A+(-)^{\epsilon_N(\epsilon_M+\epsilon_C)}
{e_M}^C{\omega_{NC}}^A\right),\\
\delta\omega_A=\delta z^N{e_N}^B{\rm D}_B\omega_A-\partial_A (\delta
z^N{e_N}^B\omega_B).\end{array}
\eqno{(21)}$$
Armed with this note, we have for any variation $\delta z^N$
$$
\delta ({\dot z}^A)={\cal D}(\delta z^A)-\delta z^C{\dot z}^B{T_{BC}}^A
\eqno{(22)}$$
where ${\dot z}^A\equiv{\dot z}^N{e_N}^A$, $\delta z^A\equiv \delta z^N
{e_N}^A$ and ${\cal D}(\delta z^A)$ is a covariantized derivative
$$
{\cal D}(\xi^A)=\dot\xi^A+\xi^B{\dot z}^M{\omega_{MB}}^A.
\eqno{(23)}$$
Now, it is straightforward to check that the most general (modulo
$\alpha$-reparametrizations) transformations leaving Eq. (8)
invariant and being consistent with Eqs. (18) and (20) have the form
$$
\begin{array}{l}\delta_kz^M{e_M}^{(+)}={\dot z}^M{e_M}^+k^{(-)},\cr
\delta_kz^M{e_M}^a=0,\cr
\delta_ke=2{\rm i}e\Phi k^{(-)}{\dot z}^M{e_M}^{(+)} - e\delta_kz^M
{e_M}^{(+)}(T_{+(+)}{}^++T_{-(+)}{}^-).\end{array}
\eqno{(24)}$$

Thus, Eqs. (24) present the most general form of Siegel's $k$-symmetry
for the (1,0) superparticle in (1,0) curved superspace. Note that these
transformations contain nontrivial contributions including the torsion
superfield and don't coincide with the direct generalization of the
flat Siegel's transformations.

The equations of motion for the (1,0) superparticle in a curved
background have the form
$$
\begin{array}{l}{\cal D}(e^{-1}{\dot z}^-)+e^{-1}{\dot z}^-{\dot z}^B
{T_{B+}}^+ +e^{-1}{\dot z}^+{\dot z}^B{T_{B+}}^-=0,\\
{\cal D}(e^{-1}{\dot z}^+)+e^{-1}{\dot z}^-{\dot z}^B
{T_{B-}}^+ +e^{-1}{\dot z}^+{\dot z}^B{T_{B-}}^-=0,\\
{\dot z}^-{\dot z}^B{T_{B(+)}}^++{\dot z}^+{\dot z}^B{T_{B(+)}}^-=0,\\
{\dot z}^M{e_M}^+{\dot z}^N{e_N}^-=0\end{array}
\eqno{(25)}$$
and it is implied that the conditions (18) or (20) (in accordance with
the case) are fulfilled.

\section{Gauge algebra and the full set of (1,0) supergravity
constraints}

As was shown in Sec. 2, the full gauge algebra for the (1,0)
superparticle in flat superspace turned out to be closed. What happen
with the algebra when the model is considered in a curved background?
Introducing the following transformations with a bosonic parameter
$\xi^-$
$$
\begin{array}{l}
\delta_\xi z^M=-\xi^-{\dot z}^+{\dot z}^-{e_-}^M,\\
\delta_\xi e=\xi^-e^2{\cal D}(e^{-1}{\dot z}^+)+\xi^-e{\dot z}^+{\dot z}^-
{T_{+-}}^+ +\\
+\xi^-e{\dot z}^+{\dot z}^{(+)}{T_{(+)-}}^- + \xi^-e{\dot z}^+{\dot
z}^+{T_{+-}}^-,\end{array}
\eqno{(26)}$$
it is straightforward to check that the commutator of two
$k$-transforma\-tions has the form
$$
[\delta_{k_1},\delta_{k_2}]=\delta_\alpha +\delta_{k_3}+\delta_\xi ,
\eqno{(27)}$$
\begin{eqnarray*}
\alpha &=& 2{\rm i}{\dot z}^+k_2^{(-)}k_1^{(-)}\Phi ,\qquad \xi^- =
2{\rm i}k_2^{(-)}k_1^{(-)}\Phi ,\\
k_3^{(-)}&=&2{\rm i}k_2^{(-)}k_1^{(-)}{\dot z}^{(+)}\Phi - 2 k_2^{(-)}
k_1^{(-)}{\dot z}^+{T_{+(+)}}^+ -\\
{}&-& k_2^{(-)}k_1^{(-)}{\dot z}^+{T_{(+)(+)}}^{(+)}.
\end{eqnarray*}
To get Eq. (27) it is necessary to use the constraints (18), (20) and
the consequences of the Bianchi identities (which are solved in the
presence of (18), (20))
$$
\begin{array}{l}
2{\rm D}_{(+)}[{T_{+(+)}}^++{T_{-(+)}}^-]+2{\rm i}\partial_+\Phi +2{\rm
i}\Phi [{T_{+-}}^-+2{T_{+(+)}}^{(+)}]+\\
\quad{}+{T_{(+)(+)}}^{(+)}[{T_{+(+)}}^+ +
{T_{-(+)}}^-]=0,\\
\partial_-\Phi +\Phi [{T_{+-}}^++2{T_{-(+)}}^{(+)}]=0,\\
\partial_{(+)}\Phi +\Phi [{T_{+(+)}}^++{T_{(+)(+)}}^{(+)}]=0.
\end{array}
\eqno{(28)}$$
Taking into account Eqs. (25), it is easy to check that the
transformations (26) are trivial symmetry of the action (8) without
imposing of new constraints on background. The rest commutators in the
algebra are
$$
\begin{array}{l}[\delta_{\xi_1},\delta_{\xi_2}]=\delta_{\xi_3},\qquad \xi_3^-=
(\xi_1^-\dot\xi_2^--\xi_2^-\dot\xi_1^-){\dot z}^+,\cr
[\delta_k,\delta_\xi ]=\delta_{k_1}+\delta_{\xi_1},\cr
k_1^{(-)}=\xi^-{\dot z}^+{\dot z}^-k^{(-)}[{T_{(+)-}}^{(+)}-
{T_{+-}}^+],\cr
\xi_1^-=2{\rm i}\Phi\xi^-k^{(-)}{\dot z}^{(+)}-\xi^-k^{(-)}{\dot z}^+
{T_{+(+)}}^+,\end{array}
\eqno{(29)}$$
where we have used the constraints (18), (20) and the consequences of
the Bianchi identities
\begin{eqnarray*}
R_{+(+)}&=&{\rm D}_{(+)}{T_{+-}}^-+{\rm D}_+{T_{-(+)}}^-
+{T_{(+)+}}^{(+)}{T_{(+)-}}^-+{T_{(+)+}}^+{T_{+-}}^-,\\
R_{(+)-}&=&{\rm D}_{(+)}{T_{-+}}^++{\rm D}_-{T_{+(+)}}^+
+{T_{(+)-}}^{(+)}{T_{(+)+}}^+-\\
{}&-&2{\rm i}\Phi{T_{-+}}^{(+)} +{T_{(+)-}}^-{T_{-+}}^+,&(30)\cr
R_{(+)(+)}&=&2{\rm i}\Phi{T_{+-}}^--2{\rm D}_{(+)}{T_{(+)-}}^--
{T_{(+)(+)}}^{(+)}{T_{(+)-}}^-,\\
\partial_-\Phi &+& \Phi [{T_{+-}}^++2{T_{-(+)}}^{(+)}]=0.
\end{eqnarray*}
Thus, the full gauge algebra turns out to be closed and nontrivially
deformed as compared to the flat one (6).

As was noted above, the set (18), (20) is minimal one which implies
correct inclusion of interaction. In general case, we can add some
additional constraints to Eqs. (18), (20) and the resulting model will
be the superparticle in the presence of more rigid (restricted)
background. Let us examine, now, one of such possibilities. Consider
the transformations, which are direct generalization of the flat one
$$
\begin{array}{l}
\delta_kz^M{e_M}^{(+)}={\dot z}^M{e_M}^+k^{(-)},\\
\delta_kz^M{e_M}^a=0,\\
\delta_ke=2{\rm i}ek^{(-)}{\dot z}^M{e_M}^{(+)};\end{array}
\eqno{(31)}$$
$$
\begin{array}{l}\delta_\xi z^M{e_M}^-=-\xi^-{\dot z}^M{e_M}^+{\dot z}^N
{e_N}^-,\\
\delta_\xi e=\xi^-e^2{\cal D}(e^{-1}{\dot z}^M{e_M}^+).\end{array}
\eqno{(32)}$$
The requirement of the invariance of the action (8) under Eq. (31)
leads to the following constraints on a background:
$$
\begin{array}{l} {T_{(+)(+)}}^+=-2{\rm i}, \qquad {T_{(+)(+)}}^-=
{T_{+(+)}}^- = {T_{-(+)}}^+ =0,\\
{T_{+(+)}}^+ + {T_{-(+)}}^-=0\end{array}
\eqno{(33)}$$
while invariance under Eq. (32) implies the conditions
$$
{T_{(+)-}}^- = {T_{(+)-}}^+ = {T_{+-}}^+ = {T_{+-}}^- =0.
\eqno{(34)}$$
Taking into account the symmetry properties of the torsion, the
constraints (33) and (34) can be written in the following compact form
$$
{T_{(+)(+)}}^a = -2{\rm i}\delta^{a+}, \qquad {T_{ab}}^c = {T_{a(+)}}^b
=0.
\eqno{(35)}$$
Because the conditions (18), (20) are present among Eqs. (35), the
model (8), (31) and (32) is the superparticle. Using the Bianchi
identities, it is easy to show that the set (35) is equivalent
to the full set of (1,0) supergravity constraints [12]. In the
presence of Eqs. (35) the equations of motion for the model are written
in the form
$$
\begin{array}{l}{\dot z}^M{e_M}^+{\dot z}^N{e_N}^-=0,\qquad {\cal D}
(e^{-1}{\dot z}^N{e_N}^+)=0,\\
{\dot z}^N{e_N}^-{\dot z}^M{e_M}^{(+)}=0, \qquad {\cal D}
(e^{-1}{\dot z}^N{e_N}^-)=0,\end{array}
\eqno{(36)}$$
and, consequently, the $\xi^-$-symmetry is absent on-shell just as in
flat case. The full gauge algebra of the theory turns out to be closed
and not deformed as compared to the flat one
$$
\begin{array}{ll}[\delta_{k_1}\delta_{k_2}]=\delta_\alpha +
\delta_{k_3} + \delta_\xi, &\quad \alpha = 2{\rm
i}k_2^{(-)}k_1^{(-)}{\dot z}^+,\cr
{}&\quad k_3^{(-)}=2{\rm i}k_2^{(-)}k_1^{(-)}{\dot z}^{(+)},\cr
{}&\quad \xi^-=2{\rm i}k_2^{(-)}k_1^{(-)}\cr
[\delta_k\delta_\xi ]=\delta_{\xi_1}, &\quad
\xi_1^-=2{\rm i}k^{(-)}\xi^-{\dot z}^{(+)}\cr
[\delta_{\xi_1},\delta_{\xi_2}]=\delta_{\xi_3}, &\quad
\xi_3^-=(\xi_1^-\dot\xi_2^--\xi_2^-\dot\xi_1^-){\dot z}^+.\end{array}
\eqno{(37)}$$
Thus, the full set of supergravity constraints follows from the
requirement that the direct generalization of the flat algebra (31), (32)
is realized in the model. In this case, the algebra is not deformed as
compared to the flat one. One can note the similarity of this approach
with the light-like integrability conditions of Ref. [9].

\section{Conclusion}

In the present paper we have considered the (1,0) superparticle model
in (1,0) curved superspace. It was shown that correct inclusion of
interaction implies some set of constraints on the background which, in
general case, doesn't coincide with the full set of (1,0) supergravity
constraints. Lagrangian gauge transformations which are consistent with
these constraints contain nontrivial contributions including the torsion
superfield and don't coincide with the direct generalization of the
flat gauge transformations to a curved background. Gauge algebra of the
theory was shown to be closed and nontrivially deformed as compared to
the flat one. The requirements leading to the full set of (1,0)
supergravity constraints were proposed.

As was shown in Sec. 2, full gauge algebra of the theory turned out to
be closed. In the case of another dimensions the algebra is open. In
the first order formalism, however, the algebra turns out to be closed
again [13], and one can directly generalize the analysis of the
present paper to the case of another dimensions. For example, in the
case of (1,1) superspace the situation looks quite analogously and the
results will be published later. We hope as well, that similar analysis
can be fulfilled for the superstring case too.

\bigskip

\centerline{Acknowledgments}

\bigskip

This work is supported in part by ISF Grant No M2I000 and
European Community Grant No INTAS-93-2058.

\bigskip

\centerline{References}

\bigskip

\noindent
1. L. Brink and J. Schwarz, Phys. Lett. B {\bf 100}, 310 (1981).\\
2. M. Green and J. Schwarz, Phys. Lett. B {\bf 136}, 367 (1984).\\
3. E. Witten, Nucl. Phys. B {\bf 266}, 245 (1986).\\
4. J.A. Shapiro and C.C. Taylor, Phys. Lett. B {\bf 181}, 67 (1986);
{\bf 186}, 69 (1987).\\
5. M.T. Grisaru, P. Howe, L. Mezincescu, B.E.W. Nilsson and P.K.
Townsend, Phys. Lett. B {\bf 162}, 166 (1985).\\
6. J.J. Atick, A. Dhar and B. Ratra, Phys. Lett. B {\bf 169}, 54
(1986).\\
7. E. Bergshoeff, E. Sezgin and P.K. Townsend, Phys. Lett. B {\bf 169},
191 (1986).\\
8. W. Siegel, Phys. Lett. B {\bf 128}, 397 (1983).\\
9. E. Bergshoeff, P.S. Howe, C.N. Pope, E. Sezgin and E. Sokatchev,
Nucl. Phys. B {\bf 354}, 113 (1991).\\
10. A.A. Deriglazov, Int. J. Mod. Phys. A {\bf 8}, 1093 (1993).\\
11. L. Brink, M. Henneaux and C. Teitelboim, Nucl. Phys. B {\bf 293},
505 (1987).\\
12. M. Evans, J. Louis and B.A. Ovrut, Phys. Rev. D {\bf }, 3045
(1987).\\
13. G. Sierra, Class. Quant. Grav., {\bf 3}, L67 (1986).
\end{document}