
| Algorithm 1: R1-NES(n)   |
|--------------------------|

and 1 while *not terminate* do

![0_image_0.png](0_image_0.png)

2 for *i = 1 to n* do 3 yi ←N (0, I) 4 zi ←N (0, 1)
5 xi ←e λ(yi + ziu) //generate sample 6 fitness[i] ←f(µ + xi)
7 end 8 Compute the natural gradient for µ, λ, u, c, and v according to Eq.2, 5, 6, 7, and 8, and combine them using Eq.1 9 µ ←µ + ηO˜ µJ
10 λ ←λ + ηO˜ λJ
11 if O˜ c log p (x|θ) < 0 then 12 c ←c + ηO˜ cJ
13 v ← v+ηO˜vJ
kv+ηO˜vJk 14 u ←e cv 15 else 16 u ←u + ηO˜ uJ //additive update 17 c ←log kuk 18 v ← u kuk 19 end 20 end

δv =u + δu

![0_image_1.png](0_image_1.png)

![0_image_2.png](0_image_2.png)

1 + 2δu>u
(u>u)
'

![0_image_3.png](0_image_3.png)

(u>u)
2 u>u
The natural gradient on c and v is given by letting δu ∝ O˜u log p (x|θ), thanks to the invariance property:
O˜ c log p (x|θ) = r
−1O˜u log p (x|θ)
>v (7)

$$=r^{-1}$$
$\mid\bar{\nabla}\mid$  . 
O˜ v log p (x|θ) = r
−1hO˜u log p (x|θ) −
O˜u log p (x|θ)
>v v i, (8)