Using the log-likelihood trick, the gradient w.r.t. the parameters can be written as

$$\begin{array}{l}{{\nabla_{\theta}J=\nabla_{\theta}\int f\left(x\right)\pi\left(x|\theta\right)d x}}\\ {{\quad\quad=\int f\left(x\right)\pi\left(x|\theta\right)\frac{\nabla_{\theta}\pi\left(x|\theta\right)}{\pi\left(x|\theta\right)}d x}}\\ {{\quad\quad=E\left[f\left(x\right)\nabla_{\theta}\log\pi\left(x|\theta\right)\right],}}\end{array}$$

from which we obtain the Monte-Carlo estimate

$$\nabla_{\theta}J\simeq\frac{1}{n}\sum_{i=1}^{n}f\left(x_{i}\right)\nabla_{\theta}\log\pi\left(x_{i}|\theta\right).$$

of the search gradient. The key step of NES then consists of replacing this gradient by the natural gradient O˜ θJ = F
−OθJ,

where
$F=E\left[\nabla_{\theta}\log\pi\left(x_{i}|\theta\right)\nabla_{\theta}\log\pi\left(x_{i}|\theta\right)^{\top}\right]$. 
is the Fisher information matrix (See Fig.1 for an illustration). This leads to a straightforward scheme of natural gradient ascent for iteratively updating the parameters

$$\theta\leftarrow\theta+\eta\bar{\nabla}_{\theta}J$$
$$=\theta+\frac{\eta}{n}\sum_{i=1}^{n}f\left(x_{i}\right)F^{-}\,\triangledown_{\theta}\log\pi\left(x_{i}|\theta\right)$$  $$=\theta+\frac{\eta}{n}\sum_{i=1}^{n}f\left(x_{i}\right)\bar{\triangledown}_{\theta}\log\pi\left(x_{i}|\theta\right).\tag{1}$$

The sequence of 1) sampling an offspring population, 2) computing the corresponding Monte Carlo estimate of the gradient, 3) transforming it into the natural gradient, and 4) updating the search distribution, constitutes one iteration of NES.

The most difficult step in NES is the computation of the Fisher information matrix with respect to the parameterization. For full Gaussian distribution, the Fisher can be derived analytically [4,16]. However, for arbitrary parameterization of C, the Fisher matrix can be highly non-trivial.

## 3 Natural Gradient Of The Rank-One Covariance Matrix Approximation

In this paper, we consider a special formulation of the covariance matrix C = σ 2I + uu>,