\section{Introduction}
It is common in engineering, economics, statistics, and machine learning  to try to minimize  a function for which no analytical form is readily accessible, and  only a zeroth-order oracle giving the function value at a given point is available. Sometimes an analytical form for the function exists, but the gradient does not have an explicit expression or it is infeasible to obtain. Specific examples of both scenarios are highlighted in the first chapter of \cite{conn2009introduction}, to which we add reinforcement learning \cite{salimans2017evolution,NIPS2018_7451,pmlr-v80-choromanski18a,MR2298287}.
When only function evaluations are available the options for optimization are somewhat limited. Some of the earliest theoretical analysis of zeroth order optimization was done on \emph{random search}  \cite{matyas1965random,rastrigin1963convergence}. In random search, a step is proposed in a randomly chosen direction and the objective function is computed; if it provides an improvement over the function value at the current position then the step is taken, otherwise a new random sample is drawn and the process is repeated.
This approach has the downside of not using derivative information, therefore relying on inexpensive function evaluations and converging slowly due to the uninformed search directions. Older methods, such as discrete gradient descent and discrete coordinate descent \cite{MR1264365,MR50243},  have similar aims but actually use approximate derivative information to determine how far to step
Extensions to functions that are differentiable almost-everywhere have been considered in \cite{gupal1977method,gupal1977algorithm}. There are alternative algorithms that behave similarly to (randomized) finite difference coordinate descent but use a random basis for choosing descent directions \cite{MR499560,nesterov2017random,duchi2015optimal,MR2298287} and ample experimental results showcase the practical utility of these approaches \cite{salimans2017evolution,NIPS2018_7451,pmlr-v80-choromanski18a}. The Itoh–Abe discrete gradient method studied in \cite{grimm2017discrete,riis2018geometric,ehrhardt2018geometric} is another approach to zeroth-order optimization, but at each iteration and for each coordinate it requires to solve a scalar equation involving the objective function. Similar considerations apply for its stochastic version, where the stepsize is still defined in an implicit way.

Our approach is based  on approximating the exact gradient by finite differences computed in a set of orthogonal random directions. Different randomized projection can be considered, recovering different approaches as special cases. Indeed, our general approach recovers  finite difference versions of coordinate descent, descent in subspaces defined by random orthogonal matrices, and spherical smoothing as special cases. All these methods can be treated in a unified way with our approach.  Our main contribution is  proving convergence results as well as convergence rates. For convex objectives,  we study convergence  in  function value and give realizable  conditions for  convergence of the iterates. Beyond convexity, we consider the case when  the objective satisfies the Polyak-Łojasiewicz (PL) condition for which we can give stronger guarantees with faster rates.  The PL condition is necessarily satisfied by functions that are strongly convex in which case our results are trivially extended to convergence of the iterates to the unique minimizer. For many   instances of our general algorithm the derived results are new. Simple experiments are also provided to show that the rates described by the theorems are achievable in practice.

The  paper is organized as follows. In Section~\ref{sec:setting}, we describe the general setting,   proposed approach and its special cases. In Section~\ref{sec:main}, we summarize and discuss our main results.
In Section~\ref{sec:pre}, we present useful preliminary  results.
In Section~\ref{Sect: Convex case} and \ref{Sect: PL}, we detail and prove the results for the convex case and for objective functions satisfying a PL condition - respectively. In Section~\ref{sec:num} we present numerical experiments and conclude in Section~\ref{sec:conc} with some remarks and open questions.

\section{Problem statement}\label{sec:setting}
We consider the problem of finding a point $x_*\in\R^d$ such that $f(x_*)= f_*$, where
\begin{equation}\label{eq:P}
\tag{P}
f_* := \inf_{x\in \R^d} f(x).
\end{equation}
The function $f\colon \  \R^d\to\R$ satisfies the following hypotheses:\\
\begin{enumerate}[label={\rm ({\bf H.\arabic{*}})}, ref={\rm\bf H.\arabic{*}}, leftmargin=4em]
\item \label{H_lip} $f_*>-\infty$, $f$ is differentiable and $\nabla f$ is $\lambda$-Lipschitz; that is, for every $x_1$ and $x_2\in\R^d$,
$$\norm{\nabla f(x_2)-\nabla f(x_1)}{}\leq \lambda \norm{x_2-x_1}{}.$$
\item \label{H_existence} $f$ is convex and has a minimizer in $\R^d$.
\end{enumerate}

The goal of this paper is to study the convergence properties of the stochastic iterative procedure described in Algorithm~\ref{eqn: derivative-free}, designed to solve numerically problem \eqref{eq:P}.
This Algorithm is based on the finite difference approximations
of a subset of directional derivatives of $f$ randomly chosen at each iteration. More precisely, the steps of the algorithm are summarized as follows: \\

\begin{algorithm}[H]
Let $x_0\in\R^d$, let $(\alpha_k)_{k\in\mathbb{N}}$ and $(h_k)_{k\in\mathbb{N}}$ be sequences of positive real values and let $(P_k)_{k\in \mathbb{N}}$ be a sequence of independent random  matrices in $\mathbb{R}^{d\times\ell}$ defined on the probability space $(\Omega,\mathcal{A},\mathbb{P})$. \\
\For{$k=0,1,\ldots$  }
{$x_{k+1} = x_k - \alpha_k P_k \nabla_{\left(P_k,h_k\right)}  f(x_k).$}
\caption{Zeroth order stochastic subspace algorithm}\label{eqn: derivative-free}
\end{algorithm}

We briefly introduce the notation and explain the main ideas behind the algorithm. Given $\ell\leq d$, a matrix $P\in\R^{d \times \ell}$ and an index $j\in \left[\ell\right]:=\left\{1,\ldots,\ell\right\}$, we let $p^{(j)}\in\R^d$ denote the $j$-th column of $P$. For $h>0$, define the vector $\nabla_{\left(P,h\right)} f(x)\in \R^{\ell}$ with entries
\begin{equation}\label{surrogate}
\left[\nabla_{\left(P,h\right)} f(x)\right]_j : = \frac{f(x+h p^{(j)})-f(x)}{h},\quad\quad \forall j\in \{1,\ldots,\ell\}.
\end{equation}
Note that $\nabla_{\left(P,h\right)} f(x)$ is the finite difference approximation of the directional derivatives of $f$ in the directions identified by the columns of $P$. Since $f$ is differentiable,
$\lim\limits_{h\to 0^+} \left[\nabla_{\left(P,h\right)}f(x)\right]_j  = \langle \nabla f(x), p^{(j)} \rangle$ for every $j\in\left[ \ell \right]$ and so
$$\lim\limits_{h\to 0^+} \nabla_{\left(P,h\right)} f(x) = P^\top \nabla f(x).$$
Using the above notation, Algorithm \ref{eqn: derivative-free} can be re-written more explicitly  as
\begin{equation*}
x_{k+1}=x_k-\alpha_k \sum_{j=1}^{\ell} \frac{f(x_k+h_k p_k^{(j)})-f(x_k)}{h_k} \ p_k^{(j)}.
\end{equation*}
The recursion has the same structure as classical gradient descent, but the gradient at $x_k$ is computed with two different forms of  approximations. On the one hand, the directional derivatives are replaced by finite differences, no derivative of the function $f$ is required. On the other hand, only the $\ell$ directions defined by the columns of $P_k$ are used. Note that $\ell$ may be smaller than the dimension of the ambient space $d$. Moreover, the directions are chosen randomly at each iteration and are not necessarily drawn from the canonical basis. Indeed, throughout the paper the following properties are the only ones  required on the random matrices $P_k$: for every $k\in\N$,
\begin{enumerate}[label={\rm ({\bf P.\arabic{*}})}, ref={\rm\bf P.\arabic{*}}, leftmargin=4em]
		\item \label{A_Pas} $P_k^\top P_k\overset{a.s.}{=}\left(d/\ell\right)\mathbb{I}$;
			\item \label{A_PE} $\mathbb{E}P_kP_k^\top= \mathbb{I}.$
\end{enumerate}
Next, we discuss several examples of algorithms that can be derived as special cases, see also Section \ref{sect: special cases}.
\begin{example}[Coordinate descent]
If $P_k$ contains $\ell$ columns of $\I_d$ chosen uniformly at random without replacement and with random sign, say $\pm e_{k_1}, \ldots \pm e_{k_\ell}$, then $p_k^{(j)} = \pm e_{k_j}$ for $j=1,\ldots,\ell$,
and  Algorithm~\ref{eqn: derivative-free} corresponds to a discretized version of parallel block-coordinate descent.
\end{example}
\begin{example}[Spherical smoothing]
Consider $P_k = (\sqrt{d/\ell})\,Q_k\I_{d\times\ell}$, where $Q_k$ is as in the $QR$-decomposition of a matrix $Z_k = Q_kR_k \in \mathbb{R}^{d \times d}$  with $R_{k_{ii}} >0,$ and each element of $Z_k$ is drawn independently from $\mathcal{N}(0,1)$.
The matrix $\I_{d \times \ell}$ truncates $Q_k$ to its first $\ell$ columns so $Q_k\I_{d\times\ell}$ corresponds to $\ell$ columns of the random orthogonal matrix distributed according to the Haar measure on orthogonal matrices \cite{Mezzadri}.
In other words, the columns $p_k^{(j)}$ are orthogonal and distributed uniformly on the sphere for all $j$. Then, when $\ell=1$, $P_k\nabla_{(P_k,h)}f(x_k)$ is a spherical smoothing estimate of the gradient, as described in, e.g., \cite{MR2298287,berahas2019theoretical}. For the case $\ell >1$ it is more common  to sample $p^{(j)}$ independently and uniformly on the sphere \cite{berahas2019theoretical}, but in our case, to satisfy Assumptions \eqref{A_Pas} and \eqref{A_PE}, the columns of $P$ must be orthonormal, similar to \cite{Kozak2021Stochastic}.
The advantage of a matrix $P$ with orthonormal columns is discussed at length in \cite{Kozak2021Stochastic}. We remark here  that this property is a valid extension of traditional spherical smoothing to subspaces of dimension greater than one, and is required to obtain our results and connect Algorithm \eqref{eqn: derivative-free} with discrete gradient descent with $\ell=d$.
\end{example}
\begin{example}[Gradient descent on random subspaces]
If $h_k =h$ for every $k\in\mathbb{N}$ and $h \to 0$, then $\nabla_{(P_k,h_k)}f(x_k)$ becomes $P_k^\top\nabla f(x_k)$ and the algorithm reduces to
\begin{equation}\label{eqn: gradient-free}
x_{k+1} = x_k - \alpha_k P_k P_k^\top \nabla f(x_k).
\end{equation}
 For shorthand, when we reference Algorithm~\ref{eqn: derivative-free} with $h_k=0$, we are referring to the use of exact directional derivatives as in recursion~\ref{eqn: gradient-free}, which has been introduced and studied in \cite{Kozak2021Stochastic}. Our analysis allows to recover and improve on previous results for the iteration in~\eqref{eqn: gradient-free}.
 \end{example}
 \begin{example}[Gradient descent]  When $\ell=d$ we have $PP^\top = P^\top P = \I_d$. Hence,  $P$ is an orthonormal basis for $\mathbb{R}^d$, so that  discrete gradient descent and gradient descent are recovered as special cases of Algorithm \ref{eqn: derivative-free} and recursion \ref{eqn: gradient-free} respectively.
\end{example}
Before stating and discussing our main results, we add one remark.
\begin{remark}[Derivative-free optimization and automatic differentiation]
 From a practical point of view, the implementation of methods based on exact gradient computations, such as~\eqref{eqn: gradient-free},  is restricted to cases where exact directional derivatives are available. Under certain conditions, one can use automatic differentiation to obtain directional derivatives, however this requirement restricts the user to particular software, and precludes experiments wherein the function is accessed only via blackbox function evaluations, as is more common in derivative-free optimization.
 Importantly, the recursion based on exact gradient computations cannot be used when the simulations are physical (such as robotics, and many engineering examples), or when the objective can not be described by the elementary functions available to automatic differentiation software (such as in reinforcement learning).
 \end{remark}
\section{Main results}\label{sec:main}
In this section,  we summarize the main results of the paper.
We provide convergence results, explain the dependence on the discretization parameters $h_k$ and the choice of the stepsizes $\alpha_k$, and provide context for the results within the larger body of literature. The section contains two theorems. Theorem \ref{Th:main1} establishes convergence properties of Algorithm~\ref{eqn: derivative-free} in the convex case, while Theorem \ref{Th:main2} deals with objective functions satisfying the Polyak-Łojasiewicz condition \ref{H_PL} without requiring convexity. The theorems are simplified versions of the results in Sections \ref{Sect: Convex case} and \ref{Sect: PL}, where more detailed statements and proofs can be found.

\begin{theorem}[Convergence - convex case]\label{Th:main1}  Assume that  conditions \ref{H_lip} and \ref{H_existence} are satisfied and suppose that  \ref{A_Pas} and \ref{A_PE} hold. Let $\seq{x_k}$ be a random sequence generated by Algorithm~\ref{eqn: derivative-free}. Set $\Lambda:=\lambda d/\ell$.\\
\begin{itemize}
\item[(i) ] Let $0\leq h_k\leq \overline{h}$ and $\alpha_k=\alpha$ for some $\overline{h} > 0$ and $\alpha\in \left(0,1/\Lambda\right)$. Then, for explicit constants $C_1, C_2, C_3$ and $C_4$,
$$\min_{j\in[k]}\mathbb{E}(f(x_j)-f_*) \quad \leq \quad  \max\{C_1/(k+1)+C_2\overline{h}^2+C_3\overline{h}, \ C_2 \overline{h}^2+C_4\overline{h}\}.$$
\item[(ii) ] Set $h_k=1/k^r$ with $r>0$ and $\alpha_k=\alpha/k$ with $\alpha\in \left(0,1/\Lambda\right)$. Then,
\begin{equation*}
		\lim_k f(x_k) \quad  \overset{\textrm{a.s.}}{=} \quad  f_*   \ \
			\end{equation*}
		and, for an explicit constant $D_0>0$,
	\begin{equation*}
	\begin{split}
	\min_{j\in[k]} \mathbb{E}\left[f(x_j) -f_*\right] \quad  \leq \quad D_0/\ln k.
	\end{split}
	\end{equation*}
	\item[(iii) ] Set $h_k=1/k^r$ with $r>1$ and $\alpha_k=\alpha$ with $\alpha\in \left(0,1/\Lambda\right)$. Then,
	\begin{equation*}
	\lim_k f(x_k) \quad  \overset{\textrm{a.s.}}{=} \quad  f_* \ \
	\end{equation*}
	and, for an explicit constant $D>0$,
	\begin{equation*}
	\begin{split}
	\min_{j\in[k]} \mathbb{E}\left[f(x_j) -f_*\right]  \quad  \leq \quad D/k.
	\end{split}
	\end{equation*}
	Moreover, there exists a random variable $x_*$ with values in $\argmin f$ such that $x_k\overset{\textrm{a.s.}}{\to} x_*$.\\
	\item[(iv) ] Consider recursion ~\ref{eqn: gradient-free} with $\alpha_k=\alpha$ and $\alpha\in \left(0,2/\Lambda\right)$. Then
	\begin{equation*}
	f(x_k) - f_* \quad  \overset{\textrm{a.s.}}{=} \quad o\left(1/k\right) \ \
	\end{equation*}
	and there exists a random variable $x_*$ with values in $\argmin f$ such that $x_k\overset{\textrm{a.s.}}{\to} x_*$.
	\end{itemize}
\end{theorem}
\begin{proof}
The results in $(i), (ii), (iii)$ and $(iv)$ are special cases of Theorems \ref{th:convex_inexact}, \ref{th:FejerWeak}, \ref{th:FejerStrong} and \ref{th:FejerSStrong} - respectively.
\end{proof}
More detailed results - and under milder assumptions - are given in the theorems cited above. There, for instance, one can find explicit computations of the constants $C_1, C_2, C_3, C_4$ and $D$. Some comments on Theorem \ref{Th:main1} are in order. In $(i)$ we show the case in which the discretization $h_k$ does not decrease to zero. We get a stability estimate for the expectation of the function values at the best iterate, depending on the upper-bound on the error noise $\overline{h}$. Notice that we recover the rate $C_1/(k+1)$ as $\overline{h}\to 0$. On the other hand, for $\overline{h}>0$, running the algorithm beyond a  certain number of iterations does not lead to any guarantee of improvement to the best iterate. More precisely, if we consider a stopping time proportional to $1/\overline{h}$, then overall accuracy will be $O(\overline{h})$ (assuming $\overline{h}\leq 1$ for simplicity). In the rest of Theorem \ref{Th:main1} we  require $h_k\to 0$. In $(ii)$ the discretization sequence is allowed to converge to zero (polynomially but) arbitrarily slowly, but also a vanishing stepsize $\alpha_k$ is required and the results we can obtain are quite weak: the function values converge to the optimum with a logarithmic rate (in expectation and for the best iterate). Better results can be obtained if $h_k$ converges to zero fast enough, as shown in $(iii)$. In this case we gain on two different sides: first, a constant stepsize $\alpha$ can be used, which is convenient from a numerical point of view; second, we get faster convergence rates for the function values of the form $1/k$ (again in expectation and for the best iterate); third, we have a.s. convergence of the iterates. To the best of our knowledge, this is the first result showing convergence of the iterates for these types of zeroth-order methods in the general convex case; recall that special cases of our method include several well-known methods such as coordinate descent and smoothing on a sphere. Finally, for  recursion~\eqref{eqn: gradient-free} in $(iv)$, we obtain an a.s. convergence rate asymptotically faster than $1/k$ for the last iterate (and not only for the best one).  We add four remarks.

\begin{remark}\label{LipschitzConst}
    In the setting of Theorem \ref{Th:main1}, $\Lambda=\lambda d / \ell$ plays the role of the Lipschitz constant. In $(iv)$ the choice of stepsize is bounded above by the classical limit $2/\Lambda$. On the other hand, as will be evident in the proofs, $(i)$-$(iii)$ require the stepsize to be bounded above by $1/\Lambda$.
\end{remark}
\begin{remark}
The ergodic iterate $\bar{x}_k:=\left(\sum_{j=1}^{k}\alpha_j\right)^{-1}\sum_{j=1}^{k} \alpha_j x_j$ attains the same rates as the ones above for the best iterate, but it is of little practical interest. Indeed, in order to apply the algorithm, $f$ has to be evaluated at each iterate $x_k$ and so it is possible to just keep the one that achieves the minimal value function.
\end{remark}
\begin{remark}
Let $N$ denote the number of function evaluations required to perform $k$ iterations. At each iteration Algorithm \ref{eqn: derivative-free} uses \ $\ell+1$ \ function evaluations. Then $N=(\ell+1)k$ and the rates in Theorem \ref{Th:main1} $(i)$-$(iii)$ can be easily re-written in terms of function evaluations. The same observation holds for the results in Theorem \ref{Th:main2}.
\end{remark}
\begin{remark}[Comparison with previous work]
Here we compare to the results in the literature dealing with a convex objective function.
\begin{itemize}
\item The case $\ell=1$ with Gaussian sampling is considered in \cite{nesterov2011random} where one can find results similar to ours in terms of convergence rates but with worse constants.  See  Remark \ref{rmk:Nest_Comp} for a precise comparison.
\item In \cite{duchi2015optimal} a mirror-descent variant of Algorithm \ref{eqn: derivative-free} is proposed to deal with a stochastic objective, similar to the setting in  \cite{MR50243}.  A rate $O(1/\sqrt{k})$  in expectation is derived, however the results should not be compared directly to our related results in $(iii)$, as the setting is more challenging.
See Remark \ref{rmk:Duchi_Comp}.
\item In \cite{Kozak2021Stochastic} only the setting $(iv)$ is studied. Stricter assumptions are required (e.g., existence of a finite $R$ such that $$\max_{x_*}\max_x \{\norm{x-x_*} : f(x) \leq f(x_0)  \} \leq R,$$ and only convergence in expectation is provided. The special case of parallel coordinate descent has been considered in \cite{tappenden2018complexity} and  \cite{salzo2019parallel}.
\end{itemize}
\end{remark}
The  following Corollary considers the case where the algorithm is run for a finite number of iterations known a priori.
\begin{corollary}\label{cor:rate} Under the assumptions of Theorem \ref{Th:main1} $(i)$, let $K\in\mathbb{N}$ and  $\overline{h} \leq 1/K$. Then, for some constant $C$,
\begin{align}\label{eq:cor1Result1}
    \min_{j\in[K]}\mathbb{E}(f(x_j)-f_*) \leq  C/K.
    \end{align}
\end{corollary}
In particular, given a tolerance $\varepsilon>0$, it is possible to choose a number of iteration $K\in\mathbb{N}$ such that $C/K\leq \varepsilon$ and a discretization $\overline{h} \leq 1/K$, in order to get
\begin{align}\label{eq:cor1Result1}
    \min_{j\in[K]}\mathbb{E}(f(x_j)-f_*) \leq  \varepsilon.
    \end{align}
    \ \\

Next we state and discuss a second set of results derived under different assumptions on the objective function.
It is well-known that first order methods exhibit favorable convergence results for strongly convex functions. More recently, it has been proved that improved convergence rates can also be obtained when the objective function satisfies weaker geometrical assumptions that do not require convexity \cite{attouch2013convergence,karimi2016linear}. In this paper we consider the (global) Polyak-Łojasiewicz condition:

\begin{enumerate}[label={\rm ({\bf H.\arabic{*}})}, ref={\rm\bf H.\arabic{*}}, leftmargin=4em]\setcounter{enumi}{2}
\item \label{H_PL} the function $f$ is $\gamma$-PL; namely, for every $x\in\R^d$,
$$\norm{\nabla f(x)}{}^2\geq \gamma \left(f(x)-f_*\right).$$
\end{enumerate}
The main example of functions satisfying the global PL condition is the class of strongly convex functions. For more examples in the non-convex setting, see \cite{attouch2013convergence} and the numerical experiments in Section \ref{sec:num}.
We stress again that in the next result we assume \ref{H_PL} but not \ref{H_existence}; that is, we do not require convexity of the objective function.
\begin{theorem}[Convergence - PL case]\label{Th:main2} Assume that  conditions \ref{H_lip} and \ref{H_PL} are satisfied and suppose that  \ref{A_Pas} and \ref{A_PE} hold. Let $\seq{x_k}$ be a random sequence generated by Algorithm~\ref{eqn: derivative-free}. Set $\Lambda:=\lambda d/\ell$ and let $\alpha\in \left(0,2/\Lambda\right)$. Fix a constant $w\leq1$ such that $0<w<2-\Lambda \overline{\alpha}$, and define $\eta:=1-w\alpha\gamma/2$.\\
\begin{enumerate}
\item[(i') ] Set $h_k \leq \overline{h}$  for some $\overline{h}>0$ and $\alpha_k=\alpha$. Then, for an explicit constant $C_1>0$,
	\begin{equation*}
	\begin{split}
	\mathbb{E}\left[f(x_k)-f_*\right] & \quad  \leq \quad  \eta^k \left(f(x_0)-f_*\right)  +C_1 \ \overline{h}^2\left[1-\eta^k\right].
	\end{split}
	\end{equation*}
\item[(ii') ] Set $h_k=1/k^r$ with $r>0$ and $\alpha_k=\alpha$. Then, there exists a constant $C_2>0$ such that
$$\mathbb{E}\left[f(x_k)-f_*\right]\quad  \leq \quad  C_2/k^{2r}.$$
\item[(iii') ] Set $h_k=\sqrt{\eta^k/k^r}$ with $r>1$ and $\alpha_k=\alpha$. Then, for an explicit constant $C_2>0$,
$$\mathbb{E}\left[f(x_k)-f_*\right]\quad \leq \quad  C_2\eta^k.$$
\item[(iv') ] Consider recursion~\eqref{eqn: gradient-free} with $\alpha_k=\alpha$. Then,
	\begin{equation*}
	\begin{split}
	\mathbb{E}\left[f(x_k)-f_*\right] &\quad  \leq \quad \eta^k \left(f(x_0)-f_*\right).
	\end{split}
	\end{equation*}
	\end{enumerate}
\end{theorem}
\begin{proof}
The results in $(i'), (ii')$ and $(iii')$ are special cases of Theorems \ref{th:PL_lin}, \ref{th:PL_alphaconst} and \ref{th:linrate} - respectively; while the result in $(iv')$ is presented here for completeness but is already considered in \cite{Kozak2021Stochastic} - see Corollary $2.3$.
\end{proof}
	Sharper and more detailed results are given in the theorems cited above, where the reader can also find the explicit computations of the constants involved in the rates. We remark only that, for a constant stepsize $\alpha_k=1/\Lambda$, the decreasing rate in $(i')$, \ $(ii')$ and $(iii')$ is given by
$\eta=1-\gamma/(2\Lambda).$
As a general comparison with Theorem \ref{Th:main1}, note that the results in Theorem \ref{Th:main2} do not involve the best iterate but only the last one. As in the first result of Theorem \ref{Th:main1}, in Theorem \ref{Th:main2}
$(i')$ the parameter $h_k$ does not necessarily vanish and so the error produced by the finite difference discretization of the gradient does not converge to zero. This explains the substantial difference between this result and the ones in $(ii')$-$(iv')$: in the upper-bound of $(i')$, for $k\to+\infty$ the right-hand side does not vanish. We can only guarantee that the expectation of the function evaluations converges with a linear rate to a sublevel set of $f$ with value $f_*+C_1 \overline{h}^2$, also called \emph{error dominated region}. On the other hand note that, in comparison with Theorem \ref{Th:main1} $(i)$, the upper-bound does not diverge with the iterations but it remains bounded. In $(iii')$, we study the case of polynomial decay of the discretization parameter; namely, a decay of the form $h_k=1/k^r$ with $r>0$. In this case, the upper-bound is proportional to $1/k^{2r}$ and  the rate gets better for a faster decay of the discretization parameter. In $(iii')$, under the assumption of a sufficiently fast (exponential) decay of $h_k$ and with a constant stepsize $\alpha_k$, we get a linear rate of convergence to the optimal value. Finally in $(iv')$, for the recursion in \eqref{eqn: gradient-free} with $\alpha_k$ constant, we recover the linear rate already shown in \cite{Kozak2021Stochastic}.
We add three remarks.
\begin{remark}[Stepsize bounds]
    Following the same discussion as in Remark \ref{LipschitzConst}, in all the results of Theorem \ref{Th:main2} the stepsize must be bounded above by the classical limit $2/\Lambda$.
\end{remark}
\begin{remark}[Adaptivity]
An important consequence of the analysis in the previous theorems is that Algorithm~\ref{eqn: derivative-free} is \emph{adaptive}; that is, knowledge of the specific properties of $f$ is not needed to ensure the corresponding convergence results. We give an explicit example. Consider an objective for which assumptions \ref{H_lip}, \ref{H_existence} are satisfied and we run the algorithm with $h_k=1/k^r$ for $r>1$ and $\alpha_k=\alpha$ for $\alpha\in \left(0,\ell/(\lambda d)\right)$. Then the results in Theorem \ref{Th:main1} $(iii)$ hold; namely, we have convergence of the iterates, almost sure convergence of the function values and a $1/k$ rate for the best iterate in expectation. In the same setting, if the function satisfies the PL inequality ($\ref{H_PL}$), from Theorem \ref{Th:main2} $(ii')$ we get automatically a $1/k^{2r}$ rate for the last iterate in expectation.
\end{remark}

\begin{remark}[Almost sure convergence]
Under the assumptions of Theorem \ref{Th:main2} $(ii')$, we have that\\
$\mathbb{E}\left[f(x_k)-f_*\right]  \lesssim C_2 / {k^{2r}}$. 	Then,  If $r>1/2$ the right hand side is summable and, by Markov's inequality and Borel-Cantelli Lemma, $f(x_k)\overset{\textrm{a.s.}}{\to} f_*$, so that $d(x_k)\overset{\textrm{a.s.}}{\to} 0.$
	In particular, if the function $f$ has a unique minimizer $x_*$, \ $x_k\overset{\textrm{a.s.}}{\to} x_*$. Analogous reasoning holds for Theorem \ref{Th:main2} $(iii')$ and $(iv')$.
\end{remark}

Finally,  the  following Corollary considers the case where the algorithm is run for a finite number of iterations known a priori.
\begin{corollary}\label{cor:rate} Under the assumptions of Theorem \ref{Th:main2} $(i')$, let $K\in\mathbb{N}$ and  $\overline{h} \leq\eta^{K/2}$. Then,
\begin{align}\label{eq:cor1Result1}
    \mathbb{E} \left[f(x_{K}) - f_*\right]  &\leq \left(f(x_0) - f_* +C_1\right) \eta^K.
    \end{align}
    Moreover, if  $d(\cdot)$ is  the Euclidean distance of the argument to the set $\argmin_x f$, then
\begin{align}\label{eq:cor1Result1}
\nonumber    \mathbb{E} \ d(x_{K}) &\leq \frac{2}{\gamma}\left(f(x_0) - f_* + C_1\right) \eta^K .
    \end{align}
\end{corollary}
\begin{proof}
The first result follows directly from the assumption $\overline{h} \leq\eta^{K/2}$ and Theorem \ref{Th:main2} $(i')$. The second claim follows from the previous one and the following inequality, that holds  under \ref{H_PL} - see e.g. \cite{bolte2017error}: for every $x\in \R^d$,
$ d(x) \leq 2/{\gamma}\left(f(x) - f_*\right).$
\end{proof}
Equation \eqref{eq:cor1Result1} allows for a solution minimizing $f$ up to an arbitrary desired accuracy $\varepsilon>0$ in expectation. To this aim it is sufficient to choose
\[
K=\frac{\ln\Big(\varepsilon/(f(x_0) - f_* +C_1)\Big)}{\ln \eta}\quad \text{ and }\quad \overline{h}\leq \eta^{K/2}.
\]
Note that the dependence on the number of iterations is logarithmic in $\varepsilon$.  A similar observation holds for $ \mathbb{E}  d(x_{K})$. If $f$ has a unique minimizer $x_*$ (e.g. $f$ is  strongly convex) Corollary \ref{cor:rate} provides a   convergence  rate for the iterates to the minimizer.

In the next sections we provide more detailed statements and proofs of the results. We start with some useful preliminary results that are the basis for the development in later sections.

\section{Preliminaries}\label{sec:pre}
\subsection{Notation and stepsize assumptions}
For $k\in \N$, we define \\
$$f_k:=f(x_k), \quad \nabla f_k:= \nabla f(x_k), \quad \nabla_k f_k:= \nabla_{\left(P_k,h_k\right)} f(x_k).$$

Now we provide the main conditions on the parameters of the proposed algorithm. Setting $\Lambda:=\lambda d/\ell$, the assumptions we consider on the sequences of stepsizes $\seq{\alpha_k}$ and discretizations $\seq{h_k}$ are as follows:\\
\begin{enumerate}[label={\rm ({\bf A.\arabic{*}})}, ref={\rm\bf A.\arabic{*}}, leftmargin=4em]
	\item \label{A_stepwith2}  $0 < \alpha_k \leq \overline{\alpha}<2/\Lambda$;
	\item \label{A_stepwith1}
	$0 < \alpha_k \leq \overline{\alpha}<1/\Lambda$;
	\item \label{A_stepnotinell1} $(\alpha_k)\notin \ell^1$ and $(h_k)$ is bounded above by some $\overline{h}\geq 0$;
	\item \label{A_stepbb} $(\alpha_k)$ is bounded below by some $\underline{\alpha}>0$ and $(h_k)$ is bounded above by some $\overline{h}\geq 0$.
\end{enumerate}
	Note that \ref{A_stepwith1} implies \ref{A_stepwith2} while \ref{A_stepbb} implies \ref{A_stepnotinell1}.
Also note that $\Lambda$ plays a role analogous to the Lipschitz constant in  classical gradient descent, and  $\Lambda=\lambda$ when $\ell=d$.

\subsection{Auxiliary bound}
We start the analysis with an auxiliary lemma that estimates the distance between the surrogate of the gradient used in the algorithm and the projected exact gradient. For this first result we assume only Lipschitz continuity of the gradient and condition \ref{A_Pas} on the matrix $P_k$; in particular, convexity of $f$ is not needed. The upper-bound is a simple consequence of the Descent Lemma \ref{DescentLemm} and  the proof  is in  the appendix.
\begin{lemma}\label{lem} Let $f$ be a function satisfying \ref{H_lip} and $P_k\in \R^{d \times \ell}$ a random matrix satisfying \ref{A_Pas}. Then, for every $x\in\R^d$ and every $k\in\N$,
	$$\norm{\nabla_{\left(P_k, h_k\right)} f(x)-P_k^\top \nabla f(x)}{}\leq\frac{\lambda d h_k }{2\sqrt{\ell}}  \ \ \ \ \ \ \ a.s.$$
\end{lemma}

\begin{remark}[Discussion on the bias] Consider a sufficiently regular function $f: \R^d \to \R$. Fix $h>0$, $\ell=1$ and a $\R^d$-valued random vector $p$ satisfying Assumptions \ref{A_Pas}-\ref{A_PE}; namely,
$$\norm{p}\overset{\textrm{a.s.}}{=} \sqrt{d} \quad \text{and} \quad \mathbb{E}\left[pp^\top\right]=\mathbb{I}.$$
For every $x\in\R^d$, by Taylor expansion we have that
\begin{eqnarray}\label{remkkk}
   && \mathbb{E} \left[\nabla_{(p,h)}f(x)\right] =\mathbb{E}\left[\frac{f(x+hp)-f(x)}{h} \ p\right]\\
    &=&\nabla f(x)+\frac{h}{2} \ \mathbb{E}\left[\langle \nabla^2 f(x) p,p\rangle \ p\right]+\frac{h^2}{6} \ \mathbb{E}\left[\langle \left[\nabla^3 f(x)p\right] p,p\rangle \ p\right]+o(h^2).\nonumber
\end{eqnarray}
We  focus on  two fundamental choices for the distribution of $p$.\\
\begin{itemize}
    \item In the coordinate-wise framework, $p$ is distributed uniformly on the (discrete) set $(\sqrt{d}\  e_i)_{i=1}^d \ \cup \ (-\sqrt{d}\  e_i)_{i=1}^d$, where $e_i$ represents the $i$-th vector of the canonical basis. Then the first-order term in \eqref{remkkk} is zero:
\begin{eqnarray*}
\mathbb{E}\left[\langle \nabla^2 f(x) p,p\rangle \ p\right]&=& \frac{1}{2d}\sum_{i=1}^d \langle \nabla^2 f(x) \sqrt{d} \ e_i,\sqrt{d} \ e_i\rangle \ \sqrt{d} \ e_i\\
&-&\frac{1}{2d}\sum_{i=1}^d \langle \nabla^2 f(x) \sqrt{d} \ e_i,\sqrt{d} \ e_i\rangle \ \sqrt{d} \ e_i=0.
\end{eqnarray*}
\item The same holds for spherical smoothing, where $p$ is distributed uniformly on the (continuous) set given by the sphere of radius $\sqrt{d}$ and centered at the origin, that we denote by $\partial B$. Also in this case, indeed,
\begin{equation*}
\begin{split}
    \mathbb{E}\left[\langle \nabla^2 f(x) p,p\rangle \ p\right]=\int_{\partial B} \langle \nabla^2 f(x) p,p\rangle \ p \ dp = 0,
\end{split}
\end{equation*}
as we are integrating an odd function on a domain that is symmetric with respect to the origin.
\end{itemize}
So, in the sufficiently regular case analysed above, the surrogate gradient proposed by coordinate-wise and spherical smoothing is unbiased up to the first-order term in $h$. But notice also that, in both cases, the term in $h^2$ is non-zero in general and the surrogate gradient is biased. This excludes the direct applicability of many theoretical studies for general stochastic algorithms such as \cite{bertsekas2000gradient} and motivates the introduction of Lemma \ref{DescentLemm}, in which we bound a.s. the distance between the surrogate gradient and the projection of the exact one. On the other hand, the case of quadratic functions - in which both coordinate-wise and spherical smoothing are  unbiased -  deserves a  tailored convergence analysis that we leave for future work. For  refined properties of spherical smoothing and coordinate-wise descent, see \cite{berahas2019theoretical}  and Lemma 1, Remark 1 in \cite{Kozak2021Stochastic}.\\
\end{remark}

\subsection{A quasi-descent lemma}\label{sec:as}
In the next proposition and corollary we obtain an a.s. estimate for the decrease of the objective function values. This result is the fundamental tool for the analysis in Section \ref{Sect: PL}, and it is also of standalone interest.
\begin{proposition}\label{prop:as}
Assume \ref{H_lip}, \ref{A_Pas} and \ref{A_stepwith2} and let the random sequence $\seq{x_k}$ be generated by Algorithm~\ref{eqn: derivative-free}. Fix a constant $w$ such that $0<w<2-\Lambda \overline{\alpha}$ \  and $ w\leq 1$. Define
$	C  := {(\ell \Lambda^2)}/{(8\min\left(1, 2-\Lambda\overline{\alpha}-w\right))}.$
	Then, for every $k\in\mathbb{N}$,
	\begin{equation*}
	\begin{split}
	f_{k+1}-f_k & \leq  -\frac{w \alpha_k}{2} \| P_k^\top\nabla f_k \|^2  + C \alpha_k h_k^2  \ \ \ \ \ \ \ a.s.
	\end{split}
	\end{equation*}
	In particular, $f_{k+1}-f_k  \leq C \alpha_k h_k^2$ a.s.
\end{proposition}

Before showing the proof of Proposition \ref{prop:as}, we make some comments on the assumptions and conclusion and provide an important consequence under additional assumptions. First note that Proposition\ref{prop:as} requires neither \ref{A_PE} nor \ref{H_existence}-\ref{H_PL}.
The difference $f_{k+1}-f_k$ is bounded a.s. by the sum of two terms: a negative term depending on the squared norm of the projected gradient $P_k^\top\nabla f_k$ and an error term proportional to $\alpha_k h_k^2$.
\begin{remark}\label{rem:h0}
	Notice that the term mutiplying $\| P_k^\top\nabla f_k \|^2$ does not depend on $h_k$. A result analogous to Proposition~\ref{prop:as} can be obtained for recursion \eqref{eqn: gradient-free}, corresponding to the limit $h\to 0$. In this case, for every $k\in\N$,
\[
f_{k+1}-f_k \leq -\frac{w\alpha_k}{2}  \|P_k^\top\nabla f_k\|^2   \ \ \ \ \ \ \ a.s.
\]
In particular, the objective function values are a.s. decreasing; namely, $f_{k+1}\leq f_k$ a.s. This fact is used later, in the proof of Theorem \ref{th:FejerSStrong}.
\end{remark}

Finally, adding condition \ref{A_stepbb} to the hypothesis of Proposition~\ref{prop:as}, we get the following corollary.
\begin{corollary}\label{cor:as}
	Assume \ref{H_lip}, \ref{A_Pas}, \ref{A_stepwith2} and \ref{A_stepbb}. Let $\seq{x_k}$ be a random sequence generated by Algorithm~\ref{eqn: derivative-free}. Let $0<w<2-\Lambda \overline{\alpha}$ \  and $ w\leq 1$. Then, for every $k\in\mathbb{N}$,
	\begin{equation}\label{estt}
	\begin{split}
	f_{k+1}-f_k & \leq  -\frac{ w \underline{\alpha}}{2} \| P_k^\top\nabla f_k \|^2  + C \overline{\alpha} \overline{h}^2  \ \ \ \ \ \ \ a.s.
	\end{split}
	\end{equation}
\end{corollary}
\begin{proof}[Proof of Proposition \ref{prop:as}]
	For notational simplicity, we denote $P_k$ by $P$ and set $e_k:= \nabla_k f_k-P_k^\top\nabla f_k$. By Assumption \ref{A_Pas}, for every $v\in\R^{\ell}$,
\begin{equation}\label{eqfff}
\norm{Pv}{}^2\overset{\textrm{a.s.}}{=}\frac{d}{\ell}\norm{v}{}^2.
\end{equation}
By Hypothesis \ref{H_lip} combined with Descent Lemma \ref{DescentLemm} and Algorithm~\ref{eqn: derivative-free}, we obtain
\begin{equation}\label{desc}
\begin{split}
f_{k+1}-f_k & \leq \langle \nabla f_k, x_{k+1}-x_k\rangle + \frac{\lambda}{2}\| x_{k+1}-x_k\|^2\\
&= -\alpha_k \langle P^\top\nabla f_k, \nabla_k f_k\rangle + \frac{\alpha_k^2\lambda}{2}\| P\nabla_k f_k\|^2.
\end{split}
\end{equation}
For the last term,  we add and subtract $PP^{\top}\nabla f_k$ and recall the definition of $e_k$, to get
\begin{equation*}
\begin{split}
\frac{\alpha_k^2\lambda}{2}\| P\nabla_k f_k\|^2
& = \frac{\alpha_k^2\lambda}{2}\| P\nabla_k f_k-PP^\top\nabla f_k+PP^\top\nabla f_k\|^2\\
& = \frac{\alpha_k^2\lambda}{2}\| Pe_k \|^2+ \frac{\alpha_k^2\lambda}{2}\| PP^\top\nabla f_k\|^2+\alpha_k^2\lambda \langle Pe_k, PP^\top\nabla f_k\rangle.
\end{split}
\end{equation*}
Now, using again the algorithm and relation \eqref{eqfff} in inequality \eqref{desc}, we have
\begin{eqnarray*}
\begin{split}
f_{k+1}-f_k
&\leq
 -\alpha_k \langle P^\top\nabla f_k, \nabla_k f_k\rangle  \\&
 + \frac{\alpha_k^2\lambda d}{2\ell}\| e_k\|^2+ \frac{\alpha_k^2\lambda d}{2 \ell}\| P^\top\nabla f_k\|^2+\frac{\alpha_k^2\lambda d}{\ell} \langle \nabla_k f_k-P^\top\nabla f_k, P^\top\nabla f_k\rangle \\
& = \alpha_k \left( \Lambda\alpha_k-1 \right) \langle P^\top\nabla f_k,  \nabla_k f_k\rangle  + \frac{\Lambda\alpha_k^2}{2}\| e_k\|^2 - \frac{\Lambda\alpha_k^2}{2}\| P^\top\nabla f_k\|^2,
\end{split}
\end{eqnarray*}
where we recall that $\Lambda$ is defined as $\Lambda:=\lambda d/\ell$. Write $\nabla_k f_k=e_k+P^\top \nabla f_k$, to get
\begin{equation*}
\begin{split}
f_{k+1}-f_k  & \leq \alpha_k \left(\Lambda\alpha_k -1 \right) \langle P^\top\nabla f_k,  e_k+P^\top \nabla f_k \rangle \\
& + \frac{\Lambda \alpha_k^2}{2}\| e_k\|^2 - \frac{\Lambda\alpha_k^2}{2}\| P^\top\nabla f_k\|^2 \\
& = \alpha_k \left( \Lambda\alpha_k-1 \right) \langle P^\top\nabla f_k,  e_k \rangle  + \frac{\Lambda \alpha_k^2}{2}\| e_k\|^2+ \alpha_k \left( \frac{\Lambda\alpha_k}{2} -1 \right) \| P^\top\nabla f_k\|^2.
\end{split}
\end{equation*}
By Young's inequality with parameter $\tau_k$ and the estimate of $\norm{e_k}{}$ from Lemma \ref{lem},
\begin{equation*}
\begin{split}
f_{k+1}-f_k & \leq \frac{\alpha_k}{2\tau_k} \abs{\Lambda\alpha_k-1} \| P^\top\nabla f_k \|^2
+\frac{\alpha_k\tau_k}{2} \abs{\Lambda\alpha_k-1} \| e_k\|^2+ \frac{\Lambda\alpha_k^2}{2}\| e_k\|^2\\
&+ \alpha_k \left( \frac{\Lambda\alpha_k}{2} -1 \right) \| P^\top\nabla f_k\|^2 \\
& \leq -\alpha_k \left(1-\frac{\Lambda\alpha_k}{2}-\frac{1}{2\tau_k} \abs{\Lambda\alpha_k-1}\right) \| P^\top\nabla f_k \|^2\\
&+\frac{\alpha_k}{2}\left(\tau_k\abs{\Lambda\alpha_k-1} +\Lambda\alpha_k\right) \frac{\lambda^2 h_k^2 d^2}{4\ell}.
\end{split}
\end{equation*}
If $\alpha_k \leq 1/\Lambda$, choosing $\tau_k=1$, we obtain
\begin{equation}\label{uno}
\begin{split}
\alpha_k \left(1-\frac{\Lambda\alpha_k}{2}- \frac{1}{2\tau_k} \abs{\Lambda\alpha_k-1} \right)
& =\frac{\alpha_k}{2}\geq \frac{w\alpha_k}{2}
\end{split}
\end{equation}
and
\begin{equation}\label{due}
\begin{split}
\left(\omega_k\abs{\Lambda\alpha_k-1} + \Lambda\alpha_k\right) \frac{\alpha_k \lambda^2 h_k^2 d^2}{8\ell
& = \frac{\ell \Lambda^2}{8} \alpha_k h_k^2 \leq C \alpha_k h_k^2.
\end{split}
\end{equation}
Similarly, if $1/\Lambda < \alpha_k \leq \overline{\alpha} < 2/\Lambda$, choose $\tau_k=\frac{\Lambda\alpha_k-1}{2-\Lambda\alpha_k-w}$. Notice that $\tau_k>0$ because $\Lambda \alpha_k-1 >0$ and
$$0<w<2-\Lambda\overline{\alpha}\leq2-\Lambda\alpha_k.$$
Then,
\begin{equation*}
\begin{split}
\alpha_k \left(1-\frac{\Lambda\alpha_k}{2}- \frac{1}{2\tau_k} \abs{\Lambda \alpha_k-1} \right)
& =\frac{w\alpha_k}{2},
\end{split}
\end{equation*}
and, since $1-w\Lambda\alpha_k\leq 1$,
\begin{equation*}
\begin{split}
\left(\tau_k\abs{\Lambda \alpha_k-1} + \Lambda \alpha_k\right) \frac{\alpha_k\lambda^2 h_k^2 d^2}{8\ell}
& = \left(\frac{1-w\Lambda\alpha_k}{2-\Lambda\alpha_k-w}\right) \frac{\ell \Lambda^2}{8} \alpha_k h_k^2 \leq C \alpha_k h_k^2.
\end{split}
\end{equation*}

\end{proof}
\begin{remark}\label{rem:simple}
	If the stepsize satisfies $\alpha_k\leq 1/\Lambda$ for every $k\in\N$ (instead of the weaker condition $\alpha_k < 2/\Lambda$ from Assumption \ref{A_stepwith2}), it is clear from the proof that the result in Proposition \ref{prop:as} reduces to
	\begin{equation*}
	\begin{split}
	f_{k+1}-f_k & \leq  -\frac{\alpha_k}{2} \| P_k^\top\nabla f_k \|^2  + \frac{\ell \Lambda^2}{8} \alpha_k h_k^2  \ \ \ \ \ \ \ a.s.
	\end{split}
	\end{equation*}
	An analogous result holds for Corollary \ref{cor:as}.
\end{remark}

\section{Convex case}\label{Sect: Convex case}

In this section we study the  case of convex objective functions.
First, in Lemma \ref{lem:Fejer} we derive a key preliminary result
establishing a stochastic Fejér monotonicity property for our method.
Using this Lemma we derive  convergence results  for different parameter choices. In Section~\ref{sebsec:inexact} we consider the case of a non-vanishing discretization. Then,  in Section~\ref{sec:basic}, and~\ref{sec:improv} we consider two different regimes where the discretization is made finer as the iteration proceeds, considering both a more  and a less aggressive strategy. Finally, in  Section~\ref{sec:david1} we consider the limit case where no discretization is considered, namely recursion~\ref{eqn: gradient-free}.
Throughout this section we use the filtration $\mathcal{F}_k = \sigma(P_1, \ldots, P_{k-1})$.

\subsection{Fejér monotonicity}
We derive an a.s.  energy estimate for the Lyapunov sequence $\seq{\norm{x_k-x_*}^2}$, where $\seq{x_k}$ is the random sequence generated by Algorithm~\ref{eqn: derivative-free} and $x_*$ is any minimizer of  $f$.

\begin{lemma}\label{lem:Fejer}
	Assume \ref{H_lip}, \ref{H_existence} and \ref{A_Pas}, \ref{A_PE}. Let $\seq{x_k}$ be a random sequence generated by Algorithm~\ref{eqn: derivative-free}. Then, for every $k\in\N$ and $x_*\in\argmin f$,
	\begin{equation*}
\begin{split}
\mathbb{E}\left[\norm{x_{k+1}-x_*}{}^2 \ \big{|}  \mathcal{F}_k\right] - \left(1+\frac{\alpha_kh_k d^2}{2\ell\sqrt{\ell}}\right) \norm{x_{k}-x_*}{}^2 &\\
\overset{\textrm{a.s.}}{\leq} \quad D_k+\frac{2d}{\ell}\alpha_k^2 \norm{\nabla f_k}{}^2 + 2\alpha_k \langle \nabla f_k, x_*-x_{k} \rangle,
\end{split}
\end{equation*}
	where $\Lambda:=\lambda d/\ell$ and
	\begin{equation}\label{def:d}
	\begin{split}
	D_k: & = \frac{\Lambda^2}{2}\left[ d\left(\alpha_k h_k\right)^2+ \sqrt{\ell} \left(\alpha_k h_k\right)\right].
	\end{split}
	\end{equation}
\end{lemma}
Lemma \ref{lem:Fejer} is the fundamental inequality used in the remainder of this section. We combine it with another important ingredient, that is the cocoercivity of $\nabla f$ (see the Baillon-Haddad Theorem, Lemma \ref{BaillHadd}) which allows us to show that the sequence $\seq{\norm{x_k-x_*}^2}$ is stochastically Fejér monotone up to the error generated at each iteration by $D_k$. Under different assumptions on the stepsize and discretization sequences the sequence $(x_k)$ exhibits different behaviors in terms of optimization and convergence rates.
\begin{proof}
	Recall the notation $\nabla_k f_k:= \nabla_{\left(P_k,h_k\right)} f(x_k)$. \text{Lemma} \ \ref{lem} provides an estimate for $\norm{e_k}$, where $e_k:=\nabla_k f_k- P_k \nabla f(x_k)$. By Assumption \ref{A_Pas}, for every $v\in\R^{\ell}$,
	\begin{equation}\label{conseq:A_Pas}
	\norm{P_k v}^2\overset{\textrm{a.s.}}{=}\frac{d}{\ell}\norm{v}^2;
	\end{equation}
	and, by \ref{A_PE}, for every fixed $w\in\R^{d}$,
	\begin{equation}\label{conseq:A_PE}
	\mathbb{E}\left[\norm{P_k^\top w}^2 \ \big| \mathcal{F}_k\right]=\norm{w}^2.\end{equation}
	Thus, for every $x_*\in\argmin f$ and every $k\in\N$, the following equality holds a.s.
		\begin{equation*}
	\begin{split}
	& \norm{x_{k+1}-x_*}{}^2 -\norm{x_{k}-x_*}{}^2\\ \quad = \quad & \norm{x_{k+1}-x_k}{}^2 + 2 \langle x_{k+1}-x_k, x_{k}-x_* \rangle\\
	(\text{Alg.~} \ref{eqn: derivative-free}) \quad = \quad  & \alpha_k^2 \norm{P_k \nabla_k f_k}{}^2  - 2\alpha_k \langle P_k \nabla_k f_k, x_{k}-x_* \rangle\\
	= \quad & \alpha_k^2 \norm{P_k \nabla_k f_k-P_k P_k^\top \nabla f_k+P_k P_k^\top \nabla f_k}{}^2 + 2\alpha_k \langle \nabla f_k, x_*-x_{k} \rangle \\
	+ & 2\alpha_k \langle P_k P_k^\top \nabla f_k -\nabla f_k, x_*-x_{k} \rangle \\
	+ & 2\alpha_k \langle P_k \nabla_k f_k- P_k P_k^\top \nabla f_k , x_*-x_{k} \rangle.
	\end{split}
	\end{equation*}
Using the fact that $(x+y)^2\leq 2x^2 + 2y^2$ and the Cauchy-Schwarz inequality, we get
\begin{equation*}
	 \begin{split}
	 & \norm{x_{k+1}-x_*}{}^2 -\norm{x_{k}-x_*}{}^2\\
	 \leq \quad  & 2\alpha_k^2 \norm{P_k e_k}{}^2 + 2\alpha_k^2 \norm{P_k P_k^\top \nabla f_k}{}^2 + 2\alpha_k \langle \nabla f_k, x_*-x_{k} \rangle \\
	+ & 2\alpha_k \langle P_k P_k^\top \nabla f_k -\nabla f_k, x_*-x_{k} \rangle + 2\alpha_k \norm{P_k e_k}{} \norm{x_*-x_{k}}{}.
	\end{split}
	\end{equation*}
	Then, by \ref{A_Pas} and Lemma \ref{lem},
	\begin{equation}\label{ineqforerror}
	\begin{split}
    & \norm{x_{k+1}-x_*}{}^2 -\norm{x_{k}-x_*}{}^2\\
    \leq \quad & \frac{2\alpha_k^2d\left(\lambda h_k d\right)^2}{4\ell^2} + \frac{2\alpha_k^2d}{\ell} \norm{P_k^\top \nabla f_k}{}^2 + 2\alpha_k \langle \nabla f_k, x_*-x_{k} \rangle \\
	+ & 2\alpha_k \langle P_k P_k^\top \nabla f_k -\nabla f_k, x_*-x_{k} \rangle + \frac{\alpha_k \lambda h_k d^2}{\ell\sqrt{\ell}}  \norm{x_*-x_{k}}{}.
	\end{split}
	\end{equation}
	Finally,
	\begin{equation*}
	\begin{split}
	& \norm{x_{k+1}-x_*}{}^2 -\norm{x_{k}-x_*}{}^2 \\ \leq \quad & \frac{d^3 \lambda^2  }{2\ell^2}\left(\alpha_k h_k\right)^2 + \frac{2d}{\ell}\alpha_k^2 \norm{P_k^\top \nabla f_k}{}^2 + 2\alpha_k \langle \nabla f_k, x_*-x_{k} \rangle \\
	+ & 2\alpha_k \langle P_k P_k^\top \nabla f_k -\nabla f_k, x_*-x_{k} \rangle + \frac{ \lambda^2 d^2\left(\alpha_k h_k\right)^2}{2 \ell\sqrt{\ell}\alpha_k h_k} + \frac{\alpha_k h_k d^2}{2\ell\sqrt{\ell}} \norm{x_*-x_{k}}{}^2,
	\end{split}
	\end{equation*}
	where we used Young inequality with parameter $(\alpha_k h_kd^2)/(\ell\sqrt{\ell})$.
	Taking the conditional expectation given $\mathcal{F}_k$ and using Assumption \ref{A_PE}, we get the claim. For integrability considerations, see  Remark \ref{lemma: integrabilityI}.
\end{proof}

Given the above inequality, we next derive a number of different convergence results considering different choices of the discretization parameter and the stepsize.

\subsection{Non-vanishing dicretization
}\label{sebsec:inexact}

Using an intermediate estimate from the proof of Lemma \ref{lem:Fejer}, we get the following result: with only a boundedness assumption on the discretization sequences, we derive an upper bound on the expectation of the function values for the best iterate.
\begin{theorem}\label{th:convex_inexact}
	Assume \ref{H_lip}, \ref{H_existence}, \ref{A_Pas}, \ref{A_PE} and \ref{A_stepwith1}. Let $\seq{x_k}$ be a random sequence generated by Algorithm~\ref{eqn: derivative-free}. Then, for every $k\in\N$,
	\begin{equation*}
	\begin{split}
	\min_{j\in[k]}\mathbb{E}(f_j-f_*)&\leq \frac{\tilde{D}_k}{\sum_{j=0}^{k}\alpha_j},
	\end{split}
	\end{equation*}
	where
	\begin{equation*}
	\begin{split}
	    	\tilde{D}_k:&=\frac{d+\xi \ell}{2\xi\ell}\left[S_k +  \sum_{j=0}^{k}  \rho_j \left(\sqrt{S_{j-1}}+\sum_{i=0}^{j} \rho_i\right) \right],
	\end{split}
	\end{equation*}
	with $\xi:=\frac{1}{\lambda}-\frac{d\overline{\alpha}}{\ell}>0$, $\rho_i:=\frac{ \lambda d^2}{\ell\sqrt{\ell}} \left(\alpha_i h_i\right) $ and $S_j:=\norm{x_0-x_*}{}^2+\sum_{i=0}^{j}\frac{\Lambda^2}{2} \left(\alpha_i h_i\right)^2$. In particular, for $\alpha_k=\alpha$ and $h_k\leq \overline{h}$, we get
	\begin{equation*}
	\begin{split}
\min_{j\in[k]}\mathbb{E}(f_j-f_*)&\leq \max\{C_1/(k+1)+C_2\overline{h}^2+C_3\overline{h}, \ C_2 \overline{h}^2+C_4\overline{h}\},
	\end{split}
	\end{equation*}
	where we made explicit only the dependence on the iteration number $k$ and the bound on the discretization error $\overline{h}$, while $C_1, C_2$ and $C_3$ are appropriate constants derived from the proof.
\end{theorem}
The above result is an extended version of Theorem~\ref{Th:main1}$(i)$.  As mentioned after Theorem~\ref{Th:main1}, it suggests that the accuracy will stop improving after a given number of iterations depending on the discretization level. The proof follows.

\begin{proof}
Start from the inequality in \ref{ineqforerror}. Recalling that we defined $\rho_j:=\frac{ \lambda d^2}{\ell\sqrt{\ell}} \left(\alpha_j h_j\right) $, for every $j\in\N$ we have that
	\begin{equation*}
	\begin{split}
	\norm{x_{j+1}-x_*}{}^2 -\norm{x_{j}-x_*}{}^2 &  \leq \frac{\Lambda^2}{2} \left(\alpha_j h_j\right)^2 + \frac{2d}{\ell} \alpha_j^2\norm{P_j^\top \nabla f_j}{}^2 + 2\alpha_j \langle \nabla f_j, x_*-x_{j} \rangle \\
	& \ \ + 2\alpha_j \langle P_j P_j^\top \nabla f_j -\nabla f_j, x_*-x_{j} \rangle+ \rho_j \norm{x_*-x_{j}}{}\\
		\left(\text{Baillon-Haddad Theorem \ref{BaillHadd}}\right) \ &\leq \frac{\Lambda^2}{2} \left(\alpha_j h_j\right)^2 + \frac{2d}{\ell} \alpha_j\norm{P_j^\top \nabla f_j}{}^2 - \frac{2}{\lambda} \alpha_j \norm{\nabla f_j}{}^2 \\
	& \ \ + 2\alpha_j \langle P_j P_j^\top \nabla f_j -\nabla f_j, x_*-x_{j} \rangle+ \rho_j \norm{x_*-x_{j}}{}.
	\end{split}
	\end{equation*}
	Taking expectations in the previous bound and denoting $u_j:=\sqrt{\mathbb{E}\norm{x_j-x_*}^2}$, we get that for every $j\in\N$
		\begin{equation*}
	\begin{split}
	u_{j+1}^2 -u_j^2 &\leq \frac{\Lambda^2}{2} \left(\alpha_j h_j\right)^2 + \frac{2d}{\ell} \alpha_j^2\mathbb{E}\left[\norm{\nabla f_j}{}^2\right] - \frac{2}{\lambda}\alpha_j \mathbb{E}\left[\norm{\nabla f_j}{}^2\right]  + \rho_j\mathbb{E}\sqrt{\norm{x_*-x_{j}}{}^2}\\
	&\leq \frac{\Lambda^2}{2} \left(\alpha_j h_j\right)^2 - 2\xi\alpha_j \mathbb{E}\left[\norm{\nabla f_j}{}^2\right] + \rho_j u_j,
	\end{split}
	\end{equation*}
	where we recall that $\xi:=\frac{1}{\lambda}-\frac{d\overline{\alpha}}{\ell}>0$. Then, summing from $j=0$ to $j=k$,
		\begin{equation*}
	\begin{split}
	u_{k+1}^2 +2\xi\sum_{j=0}^{k}\alpha_j \mathbb{E}\left[\norm{\nabla f_j}{}^2\right]\leq \underbrace{u_0^2+\sum_{j=0}^{k}\frac{\Lambda^2}{2} \left(\alpha_j h_j\right)^2}_{=S_{k}} +\sum_{j=0}^{k} \rho_j u_j,
	\end{split}
	\end{equation*}
    and
    \begin{equation}\label{bound1}
	\begin{split}
    \sum_{j=0}^{k}\alpha_j \mathbb{E}\left[\norm{\nabla f_j}{}^2\right]\leq \frac{1}{2\xi}\left[S_{k} +\sum_{j=0}^{k} \rho_j u_j\right]
	\end{split}
	\end{equation}
	and, by discrete Bihari's Lemma \ref{Bihari}, for every $j\in\N$
	\begin{equation}\label{bound2}
	u_j\leq \frac{1}{2}\sum_{i=0}^{j}\rho_i+\left[S_{j-1}+\left(\frac{1}{2}\sum_{i=0}^{j}\rho_i\right)^2\right]^{1/2}\leq \sqrt{S_{j-1}}+ \sum_{i=0}^{j}\rho_i.
	\end{equation}
	Starting again from the intermediate inequality in \ref{ineqforerror}, for every $j\in\N$,
		\begin{equation*}
	\begin{split}
	\norm{x_{j+1}-x_*}{}^2 -\norm{x_{j}-x_*}{}^2 &  \leq \frac{\Lambda^2}{2} \left(\alpha_j h_j\right)^2 + \frac{2d}{\ell} \alpha_j^2\norm{P_j^\top \nabla f_j}{}^2 + 2\alpha_j \langle \nabla f_j, x_*-x_{j} \rangle \\
	& \ \ + 2\alpha_j \langle P_j P_j^\top \nabla f_j -\nabla f_j, x_*-x_{j} \rangle+ \rho_j \norm{x_*-x_{j}}{}\\
		\left(\text{convexity}\right) \ &\leq \frac{\Lambda^2}{2} \left(\alpha_j h_j\right)^2 + \frac{2d}{\ell} \alpha_j\norm{P_j^\top \nabla f_j}{}^2 - 2\alpha_j (f_j-f_*) \\
	& \ \ + 2\alpha_j \langle P_j P_j^\top \nabla f_j -\nabla f_j, x_*-x_{j} \rangle+ \rho_j \norm{x_*-x_{j}}{}.
	\end{split}
	\end{equation*}
	Taking expectations, we get
		\begin{equation*}
	\begin{split}
	u_{j+1}^2 -u_j^2 +2\alpha_j \mathbb{E}(f_j-f_*)
	&\leq \frac{\Lambda^2}{2} \left(\alpha_j h_j\right)^2 + \frac{2d}{\ell} \alpha_j\mathbb{E}\left[\norm{ \nabla f_j}{}^2\right] + \rho_j u_j.
	\end{split}
	\end{equation*}
	Summing from $j=0$ to $j=k$, we conclude the first claim:
		\begin{equation*}
	\begin{split}
	u_{k+1}^2 +2 \sum_{j=0}^{k}\alpha_j\mathbb{E}(f_j-f_*)&\leq S_{k} + \frac{2d}{\ell}\sum_{j=0}^{k} \alpha_j\mathbb{E}\left[\norm{ \nabla f_j}{}^2\right] + \sum_{j=0}^{k}  \rho_j u_j\\
	\eqref{bound1} \quad &\leq S_{k} + \frac{d}{\xi\ell}\left[S_{k} +\sum_{j=0}^{k} \rho_j u_j\right] + \sum_{j=0}^{k}  \rho_j u_j\\
	&= \frac{d+\xi \ell}{\xi\ell}\left[S_{k} +  \sum_{j=0}^{k}  \rho_j u_j\right]\\
	\eqref{bound2} \quad &\leq \frac{d+\xi \ell}{\xi\ell}\left[S_{k} +  \sum_{j=0}^{k}  \rho_j \left(\sqrt{S_{j-1}}+\sum_{i=0}^j \rho_i\right) \right]\\
	&= 2\tilde{D}_{k}.
	\end{split}
	\end{equation*}
For the second claim, for $\alpha_k=\alpha$ and $h_k\leq \overline{h}$, define $\overline{\rho}:=\frac{ \lambda d^2}{\ell\sqrt{\ell}} \left(\alpha \overline{h}\right)$ and note that
\begin{equation*}
	\begin{split}
	    	\tilde{D}_k & \leq \frac{d+\xi \ell}{2\xi\ell}\left[\norm{x_0-x_*}{}^2+\sum_{j=0}^{k}\frac{\Lambda^2}{2} \left(\alpha_j h_j\right)^2 +  \norm{x_0-x_*}{}\sum_{j=0}^{k}  \rho_j\right.\\
	       & \quad \quad \left. +\sum_{j=0}^{k}\rho_j\left(\sum_{i=0}^{j}\frac{\Lambda}{\sqrt{2}} \left(\alpha_i h_i\right)\right)  +\sum_{j=0}^{k}  \rho_j\left(\sum_{i=0}^{j} \rho_i \right)\right]\\
	    	& \leq \frac{d+\xi \ell}{2\xi\ell}\left[\norm{x_0-x_*}{}^2+\frac{\Lambda^2}{2} \alpha^2 \overline{h}^2\sum_{j=0}^{k}1 + \overline{\rho} \norm{x_0-x_*}{}\sum_{j=0}^{k} 1\right. \\
	    	& \quad \quad \left. +\frac{\Lambda}{\sqrt{2}} \alpha \overline{h}\overline{\rho} \sum_{j=0}^{k}\sum_{i=0}^{j} 1+\overline{\rho}^2\sum_{j=0}^{k} \sum_{i=0}^{j} 1 \right]\\
	    	& \leq \tilde{C}_1+\tilde{C}_2\overline{h}^2(k+1)+\tilde{C}_3\overline{h}(k+1) +\tilde{C}_4\overline{h}^2(k+1)(k+2),

	\end{split}
	\end{equation*}
	where $\tilde{C}_1, ..., \tilde{C}_4$ are appropriate constants. Recalling that $\alpha_k$ is assumed to be constant and so that $\sum_{j=0}^k\alpha_j=\alpha (k+1)$, by trivial manipulations we get that
		\begin{equation*}
	\begin{split}
	\min_{j\in[k]}\mathbb{E}(f_j-f_*)&\leq \frac{\bar{C}_1}{k+1}+\bar{C}_2 (h+\overline{h}^2)+\bar{C}_3 \overline{h}^2 k,
	\end{split}
	\end{equation*}
	where $\bar{C}_1, \bar{C}_2$ and $\bar{C}_3$ are again appropriate constants. We conclude simply by noticing that, for $k\leq 1/\overline{h}$, the right hand side is bounded by $\bar{C}_1/(k+1)+\bar{C}_2 (h+\overline{h}^2)+\bar{C}_3 \overline{h}$; while, for $k>1/\overline{h}$, the sequence $\min_{j\in[k]}\mathbb{E}(f_j-f_*)$ is non-increasing in $k$ and so controlled by the bound at $k=1/\overline{h}$, that is $\bar{C}_1/(1/\overline{h}+1)+\bar{C}_2 (h+\overline{h}^2)+\bar{C}_3 \overline{h}\leq (\bar{C}_1+\bar{C}_2+\bar{C}_3)\overline{h}+\bar{C}_2 \overline{h}^2$.
\end{proof}

\begin{remark}\label{rmk:Nest_Comp0}
The case of a single random direction at each iterations $(\ell=1)$ sampled from a normal distribution is studied in \cite{nesterov2011random,nesterov2017random}. Under Hypothesis \ref{H_lip} and \ref{H_existence}, \cite[Theorem 8]{nesterov2017random} states the following result:
taking constant $\alpha_k=\frac{1}{4\lambda\left(d+4\right)}$ and $h_k=h$,
	\begin{equation}\label{eq:ourvsNest1}
	\begin{split}
	\min_{j\in [k]}\mathbb{E}\left[f(x_j)-f_{*}\right] & \leq \frac{4\lambda\left(d+4\right)\norm{x_0-x_*}{}^2}{k} + \frac{9\lambda h^2\left(d+4\right)^2}{25}.
	\end{split}
	\end{equation}
	Comparing this bound with the one obtained in Theorem~\ref{th:convex_inexact}, we see that this is  tighter. The main difference between the two approaches
	is due to the sampling  of the random direction. The one adopted in  \cite{nesterov2017random}, is such that the expectation of the finite difference approximation of the directional derivative is the gradient of a (Gaussian) smoothing of $f$, while such a property does not hold under our assumptions, and a different proof is needed.
	We will see in Remark~\ref{rmk:Nest_Comp}, that the different bounds lead to very similar results in terms of accuracy if the discretization error in our method is allowed to go to zero.

\end{remark}

Next, we develop our analysis for the case of a decreasing sequence of discretization parameters $(h_k)$  which allows for a finer  discretization and an increasingly  accurate approximation of the exact gradient.

\subsection{Basic results with coarser discretization}\label{sec:basic}

We begin considering very mild assumptions on the speed at which the discretization sequence vanishes. Using Lemma \ref{lem:Fejer}  we prove a.s. convergence for the function value of the best iterate, as well as a sublinear rate in expectation. The following result is an extended version of
 item $(ii)$ in Theorem~\ref{Th:main1}.

\begin{theorem}\label{th:FejerWeak}
Assume  \ref{H_lip}, \ref{H_existence}, \ref{A_Pas}, \ref{A_PE}, \ref{A_stepwith1} and \ref{A_stepnotinell1}. Assume also that $\seq{\alpha_k h_k}  \in \ell^1$. Let $\seq{x_k}$ be a random sequence generated by Algorithm~\ref{eqn: derivative-free}. Then,
	\begin{equation*}
	\lim_k f(x_k) \overset{\textrm{a.s.}}{=} f_*.
	\end{equation*}
	Moreover, we have the following convergence rate for the best iterate in expectation:
	\begin{equation*}
	\begin{split}
	\min_{j\in[k]}\mathbb{E}\left[f_j-f_*\right] \quad \leq \quad \frac{D}{\sum_{j=0}^{k}\alpha_j},
	\end{split}
	\end{equation*}
where the constant $D>0$ is provided in the proof.
\end{theorem}
	\begin{example}\label{ex1}
	For every $k\in\N$ let $h_k=h/k^r$ and $\alpha_k=\alpha/k^s$. Let $\alpha\in (0,1/\Lambda)$, $h>0$ and $s\geq 0$ with
 $
	r>1-s\geq 0.
$
	Then $\seq{\alpha_k}\notin \ell^1$, \ $\seq{\alpha_k h_k}\in\ell^1$, so the assumptions of Theorem \ref{th:FejerWeak} are satisfied. For example, the latter holds for  $\alpha_k=\alpha/k$ and $h_k=h/k^r$ with $r>0$ ($\alpha_k$ vanishing and $h_k$ going to zero arbitrarily slow); or for  $\alpha_k=\alpha$ and $h_k=h/k^r$ with $r>1$ ($\alpha_k$ constant and $h_k$ going to zero sufficiently fast).
\end{example}
\begin{proof
Consider the inequality from Lemma \ref{lem:Fejer}, namely
			\begin{equation}\label{ineq2}
		\begin{split}
		 \mathbb{E}_k\norm{x_{k+1}-x_*}{}^2 - & \left(1+\frac{\alpha_kh_k d^2}{2\ell\sqrt{\ell}}\right) \norm{x_{k}-x_*}{}^2\\
		\leq \ & D_k+\frac{2d}{\ell}\alpha_k^2 \norm{\nabla f_k}{}^2 + 2\alpha_k \langle \nabla f_k, x_*-x_{k} \rangle \\
		\left(\text{cocoercivity of} \ \nabla f, \text{see Thm. \ref{BaillHadd}}\right) \ \leq & \  D_k+\frac{2d}{\ell}\alpha_k^2 \norm{\nabla f_k}{}^2 - \frac{2\alpha_k}{\lambda} \norm{\nabla f_k}{}^2\\
		\left(\ref{A_stepwith1}\right) \ \leq & \  D_k+2\alpha_k\left(\frac{d\overline{\alpha}}{\ell}- \frac{1}{\lambda}\right) \norm{\nabla f_k}{}^2\\
		= & \  D_k-2\xi\alpha_k\norm{\nabla f_k}{}^2,
		\end{split}
		\end{equation}
		where we defined $\xi:=\frac{1}{\lambda}-\frac{d\overline{\alpha}}{\ell}>0$. By the assumptions, $\seq{\alpha_k h_k}\in \ell^1$ and so $\seq{D_k}\in\ell^1$.	Using Robbins-Siegmund Lemma \ref{PC}, we know that $\seq{\norm{x_{k}-x_*}{}}$ is a.s. convergent for every $x_*\in\argmin f$ and that a.s.  $$\seq{\alpha_k\norm{\nabla f_k}{}^2}\in\ell^1.$$
		By Lemma \ref{lem:Fejer},
		\begin{equation}\label{needed2}
		\begin{split}
		& \mathbb{E}_k\left[\norm{x_{k+1}-x_*}{}^2\right] - \left(1+\frac{\alpha_kh_k d^2}{2\ell\sqrt{\ell}}\right) \norm{x_{k}-x_*}{}^2\\
		\leq & \ D_k+\frac{2d}{\ell}\alpha_k^2 \norm{\nabla f_k}{}^2 + 2\alpha_k \langle \nabla f_k, x_*-x_{k} \rangle \\
		\left(\text{convexity of $f$ and } \ref{A_stepwith1}\right) \ \leq & \ D_k+\frac{2d\overline{\alpha}}{\ell}\alpha_k \norm{\nabla f_k}{}^2 -2\alpha_k\left(f_k-f_*\right).
		\end{split}
		\end{equation}
		By the Robbins-Siegmund Lemma \ref{PC}, we get that a.s.
		\begin{equation}\label{condl1}
		\seq{\alpha_k\left(f_k-f_*\right)}\in\ell^1.
		\end{equation}
		We know that $f_k\geq f_*$ and, by Assumption \ref{A_stepnotinell1}, the sequence $(\alpha_k)$ is positive and does not belong to $\ell^1$. So,
		\begin{equation}\label{liminf}
		\liminf_k f(x_k) \overset{\textrm{a.s.}}{=} f_*.
		\end{equation}
		Recall that, from Proposition \ref{prop:as}, we have that \begin{equation*}
	    \begin{split}
	    f_{k+1}-f_k & \leq C \alpha_k h_k^2  \ \ \ \ \ \ \ a.s.
	    \end{split}
	    \end{equation*}
	    By the assumptions $(\alpha_k h_k)\in\ell^1$ and $h_k$ bounded, we know that also $(\alpha_k h_k^2)$ belongs to $\ell^1$. Then, from Lemma \ref{PC_det}, $f_k$ is a.s. convergent. This implies, joint with \ref{liminf}, that $\lim_k f(x_k) \overset{\textrm{a.s.}}{=} f_*$.\\
		For the convergence rate, first take the total expectation in \eqref{ineq2},
		\begin{equation*}
		\begin{split}
		\mathbb{E} \norm{x_{j+1}-x_*}{}^2 - \left(1+\frac{\alpha_kh_k d^2}{2\ell\sqrt{\ell}}\right)\mathbb{E} \norm{x_{j}-x_*}{}^2 +  2\xi\alpha_j \mathbb{E}\norm{\nabla f_j}{}^2 &\leq D_j .
		\end{split}
		\end{equation*}
		Applying Lemma \ref{PC_det} to the deterministic sequence $\seq{\mathbb{E}\left[\norm{x_{k}-x_*}{}^2\right]}$ we get convergence for every $x_*\in\argmin f$ (and so the sequence is bounded above by some constant $C(x_*)$).
		Moreover, summing from $j=0$ to $j=k$,
		\begin{equation}\label{intermediate}
		\begin{split}
		\sum_{j=0}^k\alpha_j \mathbb{E}\norm{\nabla f_j}{}^2&\leq \frac{1}{2\xi } \sum_{j=0}^k\left( \mathbb{E} \norm{x_{j}-x_*}{}^2 -\mathbb{E}\norm{x_{j+1}-x_*}{}^2\right)\\
		& \quad + \frac{C(x_*)d^2}{4\xi\ell\sqrt{\ell}} \sum_{j=0}^k\alpha_j h_j+ \frac{1}{2\xi }\sum_{j=0}^k D_j \\
		& = \frac{1}{2\xi } \left(\mathbb{E} \norm{x_{0}-x_*}{}^2 -\mathbb{E}\norm{x_{k+1}-x_*}{}^2\right)\\
		& \quad + \frac{C(x_*)d^2}{4\xi\ell\sqrt{\ell}} \sum_{j=0}^k\alpha_j h_j+ \frac{1}{2\xi }\sum_{j=0}^k D_j \\
		& \leq \frac{1}{2\xi } \left(\norm{x_{0}-x_*}{}^2+ \frac{C(x_*)d^2}{2\ell\sqrt{\ell}} \sum_{j=0}^{+\infty}\alpha_j h_j+ \sum_{j=0}^{+\infty} D_j\right).
		\end{split}
		\end{equation}
		Taking the total expectation in inequality \eqref{needed2} and recalling that $\mathbb{E}\left[\norm{x_{k}-x_*}{}^2\right]$ is bounded above by some constant $C(x_*)$, we get that, for every $j\in \N$,
		\begin{equation*}
			\begin{split}
			\alpha_j\ \mathbb{E}\left(f_j-f_*\right) & \leq \frac{1}{2}\mathbb{E}\norm{x_{j}-x_*}{}^2-\frac{1}{2}\mathbb{E}\norm{x_{j+1}-x_*}{}^2+ \frac{C(x_*)d^2}{4\ell\sqrt{\ell}} \ \alpha_j h_j\\
			& \quad + \frac{D_j}{2}+\frac{d\overline{\alpha}}{\ell}\alpha_j \mathbb{E}\norm{\nabla f_j}{}^2.
			\end{split}
		\end{equation*}
		Summing from $j=0$ to $j=k$,
		\begin{equation*}
			\begin{split}
				& \sum_{j=0}^{k}\alpha_j \ \mathbb{E}\left(f_j -f_*\right)\\
		\leq \quad &
			\frac{1}{2}	\mathbb{E}\norm{x_{0}-x_*}{}^2-\frac{1}{2}\mathbb{E}\norm{x_{k+1}-x_*}{}^2 + \frac{C(x_*)d^2}{4\ell\sqrt{\ell}}\sum_{j=0}^{k}\alpha_j h_j \\
			& +\frac{1}{2} \sum_{j=0}^{k}D_j+\frac{d\overline{\alpha}}{\ell}\sum_{j=0}^k \alpha_j \mathbb{E}\norm{\nabla f_j}{}^2 \\
			\eqref{intermediate} \ \leq \quad & \frac{1}{2}\norm{x_{0}-x_*}{}^2+ \frac{C(x_*)d^2}{4\ell\sqrt{\ell}} \sum_{j=0}^{+\infty}\alpha_j h_j+\frac{1}{2} \sum_{j=0}^{+\infty} D_j\\
			& \quad  +\frac{d\overline{\alpha}}{2\xi\ell}\left( \norm{x_{0}-x_*}{}^2+ \frac{C(x_*)d^2}{2\ell\sqrt{\ell}} \sum_{j=0}^{+\infty}\alpha_j h_j+\sum_{j=0}^{+\infty} D_j\right)\\

			= \quad &  \frac{\xi\ell+d\overline{\alpha}}{2\xi\ell}\left(\norm{x_{0}-x_*}{}^2+ \frac{C(x_*)d^2}{2\ell\sqrt{\ell}} \sum_{j=0}^{+\infty}\alpha_j h_j\right.\\
			&\left. \quad  +\frac{\Lambda^2}{2}\sum_{j=0}^{+\infty} \left[ d\left(\alpha_j h_j\right)^2+\sqrt{\ell} \left(\alpha_j h_j\right)\right]\right)\\
			\leq \quad & \frac{1}{2\left(1-\Lambda \overline{\alpha}\right)}\left(\norm{x_{0}-x_*}{}^2+ \frac{1}{2}\left(\frac{C(x_*)d^2}{\ell\sqrt{\ell}}+\Lambda d \overline{h} +\Lambda^2\sqrt{\ell}\right) \sum_{j=0}^{+\infty}\alpha_j h_j\right)\\
			& = : D < + \infty.
			\end{split}
		\end{equation*}
		We obtain the bound by noticing that
		\begin{equation*}
			\begin{split}
				\min_{j\in[k]}\mathbb{E}\left[f_j-f_*\right] \sum_{j=0}^{k}\alpha_j & \leq \sum_{j=0}^{k}\alpha_j \ \mathbb{E}\left(f_j -f_*\right).
			\end{split}
		\end{equation*}
\end{proof}

\subsection{Improved results with finer discretization}\label{sec:improv}
 We next make stronger assumptions on the sequences $\seq{\alpha_k}$ and $\seq{h_k}$ allowing us to derive an a.s. convergence result for the function values and a.s. convergence of the iterates to a solution.
 The following result is an extended version of Theorem~\ref{Th:main1} $(iii)$.

\begin{theorem}\label{th:FejerStrong}
	Under the same conditions as in Theorem \ref{th:FejerWeak}, but with the stronger Assumption \ref{A_stepbb} instead of \ref{A_stepnotinell1}. Namely, assume \ref{H_lip}, \ref{H_existence}, \ref{A_Pas}, \ref{A_PE} and $0<\underline{\alpha}\leq\alpha_k\leq \overline{\alpha} < 1/\Lambda$ and $\seq{h_k}\in \ell^1$. Let $\seq{x_k}$ be a  sequence generated by Algorithm~\ref{eqn: derivative-free}. Then,
	\begin{equation*}
	\lim_k f_k \overset{\textrm{a.s.}}{=} f_*
	\end{equation*}
	and there exists a random variable $x_*$ with values in $\argmin f$ such that $	x_k \overset{\textrm{a.s.}}{\to} x_*.$
\end{theorem}
\begin{example}
	For every $k\in\N$, let $\alpha_k=\alpha$ constant in $\left(0,1/\Lambda\right)$ and $h_k=h/k^r$ with $h>0$ and $r>1$. Then the assumptions of Theorem \ref{th:FejerStrong} hold. These conditions are a special case of those in Example \ref{ex1}. In general, under Assumption \ref{A_stepbb} (required for Theorem \ref{th:FejerStrong}), the stepsize $\alpha_k$ is uniformly bounded below by a strictly positive constant and so it can not converge to zero. Then,  to get the condition $\seq{\alpha_k h_k}\in\ell^1$, $h_k$ can not converge to zero arbitrarily slowly as in Example \ref{ex1}. Indeed, for Theorem \ref{th:FejerStrong} to hold with $h_k$ of the form $h/k^r$, $h_k$ has to converge to zero strictly faster than $1/k$.
\end{example}
\begin{remark}\label{rmk:rates}
	Under the assumptions in Theorem \ref{th:FejerStrong}, the convergence rate in Theorem \ref{th:FejerWeak} holds and reads as
	\begin{equation*}
	\begin{split}
	\min_{j\in[k]}\mathbb{E}\left[f_j-f_*\right] \quad
	\leq  \quad \frac{D}{\underline{\alpha} k}.
	\end{split}
	\end{equation*}
\end{remark}
\begin{proof
	From the proof of the previous theorem, we see that for every $x_*\in\argmin f$  the sequence $\seq{\norm{x_{k}-x_*}{}}$ is a.s. convergent and  a.s. $\seq{\alpha_k\left(f_k-f_*\right)}\in\ell^1$. From Assumption \ref{A_stepbb}, $0<\underline{\alpha}\leq \alpha_k$ and so the sequence $\left(f_k-f_*\right)$ is non-negative and belongs to $\ell^1$ a.s. In particular,
	\begin{equation*}
	\lim_k f_k \overset{\textrm{a.s.}}{=} f_*.
	\end{equation*}
	More precisely, there is a $\bar{\Omega}\subseteq\Omega$ with $\mathbb{P}(\bar{\Omega})=1$ such that, for every $\omega \in \bar{\Omega}$,
	\begin{equation}\label{limmm}
	    \lim_k f(x_k(\omega)) = f_*.
	\end{equation}
	For $\omega\in \bar{\Omega}$, let $x_{k_j}(\omega)$ be a convergent subsequence of $x_{k}(\omega)$; say $x_{k_j}(\omega) \to x_{\infty}$. Then, by continuity of the function $f$ and the limit in \eqref{limmm},
$$f(x_{\infty}) = \lim\limits_{k} f(x_k(\omega)) = f_*.$$
Then $x_{\infty}\in\argmin f$, as it is a  minimizer of  $f$. Summarizing, there is a full measure set for which every cluster point of the random sequence $\seq{x_k}$ belongs to $\argmin f$. Finally, combining the latter result with the fact that, for every $x_*\in\argmin f$, $\seq{\norm{x_{k}-x_*}{}}$ is a.s. convergent, the stochastic version of Opial's Lemma \ref{StochOpial} guarantees the existence of a random variable $x_*$ with values in $\argmin f$ such that $x_k \overset{\textrm{a.s.}}{\to} x_*$.
\end{proof}
In the next remarks we compare our rates on the objective function with results available in the literature. Recall that none of the considered papers prove the convergence of the iterates.
\begin{remark}\label{rmk:Nest_Comp} We compare our results for a vanishing discretization with the ones obtained in \cite[Theorem 8]{nesterov2017random} for a single direction sampled according to a normal distribution (see also Remark~\ref{rmk:Nest_Comp0}).

	Choosing $\alpha_k=\alpha$ constant in $\left(0,1/\Lambda\right)$ and $h_k=h/k^r$ with $h>0$ and $r>1$, from Remark \ref{rmk:rates} we get that
	\begin{equation*}
	\min_{j\in[k]}\mathbb{E}\left[f(x_j) -f_*\right]  \leq  \ \ \frac{D}{\alpha k},
	\end{equation*}
	where
	$$D:=\frac{1}{2\left(1-\Lambda \alpha\right)}\left(\norm{x_{0}-x_*}{}^2+ \frac{\alpha}{2}\left(\frac{C(x_*)d^2}{\ell\sqrt{\ell}}+\Lambda d h +\Lambda^2\sqrt{\ell}\right) \sum_{j=0}^{+\infty} h_j\right).$$
For the special case $\alpha=\ell/(2\lambda d)$ and $\ell=1$ we derive (recalling that $\sum_{j=0}^{+\infty} 1/k^r =\zeta(r)<+\infty$ where $\zeta$ is the Riemann zeta function),
	\begin{equation}\label{eq:ourvsNest2}
	\min_{j\in[k]}\mathbb{E}\left[f(x_j) -f_*\right]  \leq  \frac{2\lambda d \norm{x_{0}-x_*}{}^2}{ k} + \frac{h d^2\zeta(r)}{2 k}\left(C(x_*)+ \lambda h +\lambda^2\right). \ \
	\end{equation}
Comparing equations \eqref{eq:ourvsNest1} and \eqref{eq:ourvsNest2} we see that the dependence on the dimension is
the same however our result converges to the optimum because we chose a decreasing discretization parameter. In addition, we
are free to choose the stepsize bigger than the one proposed in \cite{nesterov2011random,nesterov2017random} resulting in slightly better constants.
On the other hand, in \cite{nesterov2017random}, they also study the case of accelerated inertial algorithms. A similar comparison to the one above can be done also with the results in \cite{ghadimi2013stochastic}.
\end{remark}
\begin{remark}\label{rmk:Duchi_Comp}
The minimization of a smooth function via a zeroth-order oracle is also considered in  \cite{duchi2015optimal}. The assumptions in that paper are different from ours, both in terms of properties of the objective function, as well as of the available zeroth-order oracle. Regarding the objective function,  in addition to the Lipschitz continuity of the gradient, the authors of \cite{duchi2015optimal} require more restrictive assumptions, such as boundedness of  the gradient itself on the entire feasible set, which is assumed to be compact. The zeroth-order oracle instead is more general than ours, and consists of noisy function evaluations. In their setting $G$ denotes the bound on the gradient of $f$, and $R$ is the diameter of the feasible set. With the choice
\begin{equation}
\label{eq:ourvsDuchi0}
\alpha_k=\frac{\alpha R}{2G\sqrt{d/\ell}\sqrt{k} }\qquad \textrm{and} \qquad  h_k=\frac{u G}{\lambda d^{3/2} k},
\end{equation}
for some $\alpha>0$ and $u>0$, they derive a bound of the form
\begin{equation}\label{eq:ourvsDuchi1}
	\min_{j\in[k]}\mathbb{E}\left[f(x_j) -f_*\right]  \leq  \ \ \frac{5R G\sqrt{1+d/\ell}}{\sqrt{k}} \left(
	\max\{\alpha,\alpha^{-1}\}+\frac{\alpha {u}^2}{\sqrt{k}}+\frac{u\log(2k)}{k} \right).
	\end{equation}
If we choose $\alpha_k=\ell/(2\lambda d)$ and $h_k=\frac{h}{\lambda d^{3/2} k^r}$ with $r>1$, we get
\begin{equation}\label{eq:ourvsDuchi2}
	\min_{j\in[k]}\mathbb{E}\left[f(x_j) -f_*\right]  \leq  \frac{2\lambda d \norm{x_{0}-x_*}{}^2}{\ell k} + \frac{H}{2 k}\left(\frac{\lambda^2 C(x_*) \sqrt{d}}{\ell\sqrt{\ell}}+ \frac{  h}{\sqrt{d} \ell} +\frac{\lambda \sqrt{d}} {\ell\sqrt{\ell}}\right) \ \
	\end{equation}
Comparing \eqref{eq:ourvsDuchi1} and \eqref{eq:ourvsDuchi2}, we observe that we obtain a better convergence rate, due to the fact that we consider a noise-free oracle, but our analysis leads to a worse dependence on the ratio $d/\ell$.
Since the two settings are very different the significance of the comparison is somewhat limited.

\end{remark}
Finally, in the next section, we consider the case where $h=0$. i.e. Recursion ~\eqref{eqn: gradient-free}.
\subsection{Convergence results for recursion ~\eqref{eqn: gradient-free}}\label{sec:david1}
This section covers the special case of recursion ~\eqref{eqn: gradient-free}, corresponding to the limiting case of Algorithm~\ref{eqn: derivative-free} when exact directional derivatives are available.  Lemma \ref{lem:Fejexact} provides a sharper energy estimate than the one in Lemma \ref{lem:Fejer},  which in turns  leads to the improved convergence results of Theorem~\ref{th:FejerSStrong}.  The result is an extended version of Theorem~\ref{Th:main1}$(iv)$.
\begin{lemma}\label{lem:Fejexact}
	Assume \ref{H_lip}, \ref{H_existence}, \ref{A_Pas} and \ref{A_PE}. Let $\seq{x_k}$ be a random sequence generated by recursion~\eqref{eqn: gradient-free}. Then, for every $k\in\N$ and every $x_*\in\argmin f$,
	\begin{equation*}
	\begin{split}
	\mathbb{E}\left[\norm{x_{k+1}-x_*}{}^2 \ \big{|}  \mathcal{F}_k\right] -\norm{x_{k}-x_*}{}^2 \overset{\textrm{a.s.}}{=} \frac{\alpha_k^2 d}{\ell} \|\nabla f_k\|^2 +2\alpha_k \langle \nabla f_k, x_*-x_k \rangle .
	\end{split}
	\end{equation*}
\end{lemma}
\begin{proof}
	For every $k\in\N$ and every $x_*\in\argmin f$, we have that a.s.
	\begin{equation*}
	\begin{split}
	\norm{x_{k+1}-x_*}{}^2 -\norm{x_{k}-x_*}{}^2 & = \norm{x_{k+1}-x_k}{}^2 + 2 \langle x_{k+1}-x_k, x_{k}-x_* \rangle\\
	\eqref{eqn: gradient-free} \ & = \alpha_k^2 \norm{P_kP_k^\top \nabla f_k}{}^2  - 2\alpha_k \langle P_kP_k^\top \nabla f_k, x_{k}-x_* \rangle\\
	\left(\ref{A_Pas}, \text{ see } \eqref{conseq:A_Pas}\right) \ & =\frac{\alpha_k^2 d}{\ell} \langle \nabla f_k, P_k P_k^\top  \nabla f_k \rangle + 2\alpha_k \langle P_kP_k^\top \nabla f_k, x_* -x_k\rangle.
	\end{split}
	\end{equation*}
	The claim follows  taking the conditional expectation given $\mathcal{F}_k$ and using  \ref{A_PE}.
\end{proof}
Using the estimate from the above lemma and with very mild assumptions on  $\alpha_k$, we get the following result ensuring convergence of the iterates, a rate in expectation for the function values and  an asymptotic a.s. convergence rate of the form $1/k$.
\begin{theorem}\label{th:FejerSStrong}
	Assume \ref{H_lip}, \ref{H_existence}, \ref{A_Pas}, \ref{A_PE}, \ref{A_stepwith2} and \ref{A_stepnotinell1}. Let $\seq{x_k}$ be a random sequence generated by Algorithm~\ref{eqn: gradient-free}. Then there is a random variable $x_*$ with values in $\argmin f$ such that
		\begin{equation}\label{convxk}
		x_k \overset{\textrm{a.s.}}{\to} x_*.
		\end{equation}
		Moreover, the sequence $(f_k)$ is a.s. non-increasing with $\lim_k f_k \overset{\textrm{a.s.}}{=} f_*$ and the following convergence rate in expectation holds:
		\begin{equation*}
		\begin{split}
		\mathbb{E}\left[f_k -f_*\right] & \leq D_0/\sum_{j=0}^{k}\alpha_j,
		\end{split}
		\end{equation*}
			where the constant $D_0>0$ is provided in the proof.
	Finally, if \ref{A_stepbb} also holds,
	\begin{equation}
	f_k - f_* \overset{\textrm{a.s.}}{=} o(k^{-1}).
	\end{equation}
\end{theorem}
\begin{proof}
	We recall the equality from Lemma \ref{lem:Fejexact}: for every $x_*\in\argmin f$ and every $k\in \N$,
	\begin{equation}\label{repeat}
	\begin{split}
	\mathbb{E}\left[\norm{x_{k+1}-x_*}{}^2 \ \big{|}  \mathcal{F}_k\right] -\norm{x_{k}-x_*}{}^2 \overset{\textrm{a.s.}}{=} \frac{\alpha_k^2 d}{\ell} \|\nabla f_k\|^2 +2\alpha_k \langle \nabla f_k, x_*-x_k \rangle .
	\end{split}
	\end{equation}
	By Baillon-Haddad Theorem \ref{BaillHadd}, $\nabla f$ is $1/\lambda$ co-coercive and so
		\begin{align}\label{eq: ineq}
\nonumber
	\mathbb{E}\left[\norm{x_{k+1}-x_*}{}^2 \ \big{|}  \mathcal{F}_k\right] -\norm{x_{k}-x_*}{}^2 &\overset{\textrm{a.s.}}{\leq} \frac{\alpha_k^2 d}{\ell} \|\nabla f_k\|^2 -\frac{2\alpha_k }{\lambda}\|\nabla f_k\|^2\\
	(\ref{A_stepwith2}) \ &\overset{\textrm{a.s.}}{\leq} - \frac{2-\overline{\alpha}\Lambda }{\lambda} \ \alpha_k\|\nabla f_k\|^2.
	\end{align}
	Define $\xi_0:=2-\overline{\alpha}\Lambda/\lambda$, a strictly positive quantity. Then by Robbins-Siegmund Lemma \ref{PC}, for every $x_*\in\argmin f$ the random variable $\seq{\norm{x_{k}-x_*}{}}$ is a.s. convergent and that a.s. $\seq{\alpha_k\|\nabla f_k\|^2}\in\ell^1.$ Beginning again from the equality in Lemma \ref{lem:Fejexact}, we estimate the term $2\alpha_k \langle \nabla f_k, x_*-x_k \rangle$ using the convexity of $f$ and the gradient inequality:
	\begin{equation*}
	f_k + \langle \nabla f_k, x_*-x_k \rangle \leq f_*.
	\end{equation*}
	Recalling that $\alpha_k\leq\overline{\alpha}$ by Assumption \ref{A_stepwith2}, it leads to
	\begin{equation}\label{Fejexact:ineq}
	\begin{split}
	\mathbb{E}\left[\norm{x_{k+1}-x_*}{}^2 \ \big{|}  \mathcal{F}_k\right] -\norm{x_{k}-x_*}{}^2 + 2\alpha_k \left(f_k-f_*\right
	&\overset{\textrm{a.s.}}{\leq} \frac{ \overline{\alpha} d}{\ell} \alpha_k\|\nabla f_k\|^2 .
	\end{split}
	\end{equation}
Robbins-Siegmund Lemma \ref{PC} reveals that $\seq{\alpha_k (f_k-f_*)} \in \ell^1$ a.s. Since by assumption $\seq{\alpha_k}$ is not summable, $\liminf_k (f_k-f_*) \overset{\textrm{a.s.}}{=} 0$.
	By Remark~\ref{rem:h0}, the sequence $(f_k)$ is a.s. non-increasing and bounded below by $f_*$. In particular, it is a.s. convergent and $ f_k \overset{\textrm{a.s.}}{\to} f_*.$
	Following the same reasoning as in the proof of Theorem \ref{th:FejerStrong}, there is a random variable $x_*$ with values in $\argmin f$ such that $x_k \overset{\mathrm{a.s.}}{\to} x_*$.
To obtain the convergence  rate first take the total expectation in inequality \eqref{eq: ineq} and sum from $j=0$ to $j=k$ to get
\begin{equation}\label{needed}
\begin{split}
\sum_{j=0}^k \alpha_j\mathbb{E}\|\nabla f_j\|^2& \leq \frac{1}{\xi_0}\sum_{j=0}^k\left(\mathbb{E}\norm{x_{j}-x_*}{}^2- \mathbb{E}\norm{x_{j+1}-x_*}{}^2\right)\\
& = \frac{1}{\xi_0}\left(\mathbb{E}\norm{x_{0}-x_*}{}^2- \mathbb{E}\norm{x_{k+1}-x_*}{}^2\right)\\
& \leq \frac{1}{\xi_0}\norm{x_{0}-x_*}{}^2.
\end{split}
\end{equation}
Summing \eqref{Fejexact:ineq} over $k$ and combining it with \eqref{needed}, an expectation yields
	\begin{equation*}
		\begin{split}
			\sum_{j=0}^{k}\alpha_j \ \mathbb{E}\left[f_j -f_*\right]	& \leq
			\frac{1}{2}\mathbb{E}\norm{x_{0}-x_*}{}^2-\frac{1}{2}\mathbb{E}\norm{x_{k+1}-x_*}{}^2+\frac{ \overline{\alpha} d}{2\ell} \sum_{j=0}^{k} \alpha_j\mathbb{E}\|\nabla f_j\|^2\\
			\eqref{needed} \ & \leq\frac{1}{2}\norm{x_{0}-x_*}{}^2+\frac{ \overline{\alpha} d}{2\ell\xi_0} \norm{x_{0}-x_*}{}^2\\
			& = \frac{ \ell\xi_0 + \overline{\alpha} d}{2\ell\xi_0} \norm{x_{0}-x_*}{}^2\\
			& = \frac{1}{2-\Lambda\overline{\alpha}} \norm{x_{0}-x_*}{}^2 =: D_0 < + \infty.
		\end{split}
	\end{equation*}
Since $\seq{\mathbb{E}f_j}$ is non-increasing,
$\mathbb{E}[f_k-f_*]\sum_{j=0}^k \alpha_j  \leq 	\sum_{j=0}^{k}\alpha_j \ \mathbb{E}\left[f_j -f_*\right]$.
Dividing by the sum over $\alpha_j$ yields the rate.
Finally, assuming also  \ref{A_stepbb} and using the fact that $\seq{\alpha_k (f_k-f_*)} \in \ell^1$ a.s., we get $\seq{f_k-f_*} \in \ell^1$ a.s. Since $\seq{f_k-f_*}$ is also a.s. non-increasing, we conclude by Lemma \ref{smallorate} that $f_k-f_* \overset{\textrm{a.s.}}{=} o(k^{-1})$.
\end{proof}
\begin{remark}
Note that the constant $D_0$ in Theorem \ref{th:FejerSStrong} indeed corresponds to $D$ in Theorem \ref{th:FejerWeak} when the discretization is set $h_k=0$ for every $k\in\N$.
\end{remark}

\begin{remark}
Under the same conditions of Remark \ref{rmk:Nest_Comp} but considering the analogue of recursion \eqref{eqn: gradient-free}, the following result is obtained in \cite[Theorem 8]{nesterov2017random}:
taking constant $\alpha_k=1/(4\lambda\left(d+4\right))$ and $h_k=h$,
	\begin{equation}\label{eq:ourvsNest1bis}
	\begin{split}
	\mathbb{E}\left[f(\bar{x}_k)-f_{*}\right] & \leq 4\lambda\left(d+4\right)\norm{x_0-x_*}{}^2/(2k).
	\end{split}
	\end{equation}
    Under the same assumptions, consider the case of the sampling that we proposed (see Assumptions \ref{A_Pas} and \ref{A_PE}). From Theorem \ref{th:FejerSStrong}, we get that
	\begin{equation*}
	\mathbb{E}\left[f(\bar{x}_k) -f_*\right]  \ \leq \   \frac{\norm{x_{0}-x_*}{}^2}{2\left(1-\Lambda \alpha\right)\alpha k}.
	\end{equation*}
    For the case $\alpha=\ell/(2\lambda d)$ and $\ell=1$ we derive
	\begin{equation}\label{eq:ourvsNest2bis}
	\mathbb{E}\left[f(\bar{x}_k) -f_*\right]  \leq  2\lambda d \norm{x_{0}-x_*}{}^2/k
	\end{equation}
	and the same observations of Remark \ref{rmk:Nest_Comp} hold.
\end{remark}
\section{Polyak-Łojasiewicz case}\label{Sect: PL}
In contrast with the case of a general convex $f$ considered thus far, this section assumes the PL inequality (\ref{H_PL}), but convexity (\ref{H_existence}) is not needed. Since strong convexity implies the PL inequality, all of the results in this section hold when the objective function is strongly convex. We use the PL inequality in the a.s. quasi-decreasing estimates of Section \ref{sec:as} to get the main estimate of Lemma \ref{lem:mainineq}. The application of  Lemma \ref{lem:mainineq} in different settings leads to the convergence rates in expectation for the function values given in Theorems \ref{th:PL_lin}, \ref{th:PL_alphaconst} (based on Lemma \ref{lem:aux}), \ref{th:Chung} (based on Lemma \ref{chung}) and \ref{th:linrate}.
These results are similar but intrinsically different. In Theorem \ref{th:PL_lin}, we study the case in which both sequences $\seq{h_k}$ and $\seq{\alpha_k}$ are bounded above, but not converging to zero; specifically, the error generated by the discretization does not vanish. In this context, we obtain a \emph{linear} rate in expectation not to the optimal value, but to a sublevel of the objective function depending on $\overline{C}, \underline{\tau}$ and $\gamma$, see \eqref{error}. In Theorem \ref{th:PL_alphaconst}, for the case of $\alpha_k$ constant and vanishing $h_k$, we get \emph{sublinear} rates in expectation \emph{to the optimum}. In Theorem \ref{th:Chung} we obtain similar rates assuming that both $\seq{\alpha_k}$ and $\seq{h_k}$ converge to zero polynomially. Note that the algorithm does not converge to the optimal value if $h_k$ does not vanish, even with rapid decay of $\alpha_k$, a fact that may be surprising to readers more familiar with first-order stochastic approximation algorithms though, as discussed,  it is easy to see why. Finally, in Theorem \ref{th:linrate} we show linear convergence rates to the optimal value for a fast decay of $h_k$ .

\subsection{Main estimate}
The following basic estimate will be used repeatedly.
\begin{lemma}\label{lem:mainineq}
Let $\seq{x_k}$ be generated by Algorithm~\ref{eqn: derivative-free}. Assume \ref{H_lip}, \ref{H_PL}, \ref{A_Pas}, \ref{A_PE} and \ref{A_stepwith2}. Then, for every $k\in\N$,
	\begin{equation*}
	\begin{split}
	\mathbb{E}\left[f_{k+1}-f_*\right] & \leq  \left(1-\frac{w\alpha_k\gamma}{2}\right) \mathbb{E}\left[f_{k}-f_*\right]  + C \alpha_k h_k^2.
	\end{split}
	\end{equation*}
	Assuming also \ref{A_stepbb}  and defining $\eta:=1-w\underline{\alpha}\gamma/2$, we get
	\begin{equation*}
	\begin{split}
		\mathbb{E}\left[f_{k}-f_*\right]
		& \leq \eta^k \left[\left(f_{0} -f_*\right)  + \frac{C}{\eta} \sum_{j=0}^{k-1}\frac{\alpha_j h_j^2}{\eta^j} \right].
	\end{split}
	\end{equation*}
\end{lemma}
\begin{proof}
Taking the conditional expectation given $\mathcal{F}_k$ in the a.s. inequality of Proposition \ref{prop:as}, we have
	\begin{equation*}
	\begin{split}
	\mathbb{E}\left[f_{k+1} \ \big{|}  \mathcal{F}_k\right]-f_k & \leq  -\frac{w\alpha_k}{2} \mathbb{E}\left[\| P_k^\top\nabla f_k \|^2 \ \big{|}  \mathcal{F}_k\right]  + C \alpha_k h_k^2\\

	(\ref{A_PE}) \ & =  -\frac{w\alpha_k}{2} \norm{\nabla f_k}{}^2  + C \alpha_k h_k^2\\
	(\ref{H_PL})  \ & \leq  -\frac{w\alpha_k}{2} \gamma \left(f_k-f_*\right)  + C \alpha_k h_k^2.
	\end{split}
	\end{equation*}
	An expectation yields the first claim. Under Assumption \ref{A_stepbb}, the last inequality yields
    \begin{equation*}
	\begin{split}
	\mathbb{E}\left[f_{k+1}-f_*\right] & \leq  \left(1-\frac{w\underline{\alpha}\gamma}{2}\right) \mathbb{E}\left[f_{k}-f_*\right]  + C \alpha_k h_k^2.
	\end{split}
	\end{equation*}
	Iterating leads to the second claim:
	\begin{equation*}
	\begin{split}
		\mathbb{E}\left[f_{k}-f_*\right]
		& \leq  \eta^k\left(f_{0} -f_*\right)\\
		&+ C \left[ \alpha_{k-1}h_{k-1}^2+\eta \alpha_{k-2}h_{k-2}^2+\eta^2 \alpha_{k-3}h_{k-3}^2+...+\eta^{k-1}\alpha_0 h_{0}^2\right]\\
	& =  \eta^k \left(f_{0} -f_*\right)  + C \sum_{j=0}^{k-1}\eta^{k-1-j}\alpha_j h_j^2\\

	& =  \eta^k \left[\left(f_{0} -f_*\right)  + \frac{C}{\eta} \sum_{j=0}^{k-1}\frac{\alpha_j h_j^2}{\eta^j} \right].
	\end{split}
	\end{equation*}
\end{proof}
\subsection{Linear quasi-rate}
We first provide an extended version of Theorem~\ref{Th:main2}$(i')$. In this case both $\seq{\alpha_k}$ and $\seq{h_k}$ are bounded above but not vanishing, leading to the following result as a direct consequence of Lemma \ref{lem:mainineq}. In particular, the following bound suggests to stop iterating when $(f_k - f_*) \leq C \overline{\alpha}\overline{h}^2/(1-\eta)$.
\begin{theorem}\label{th:PL_lin}
	Let $\seq{x_k}$ be generated by Algorithm~\ref{eqn: derivative-free}. Assume \ref{H_lip}, \ref{H_PL}, \ref{A_Pas}, \ref{A_PE}, \ref{A_stepwith2} and \ref{A_stepbb}. Then, for every $k\in\N$,
	\begin{equation*}
	\begin{split}
	\mathbb{E}\left[f_{k}-f_*\right] & \leq \eta^k \left(f_0-f_*\right)  + \frac{C \overline{\alpha}\overline{h}^2}{1-\eta}\left[1-\eta^k\right],
	\end{split}
	\end{equation*}
	where the constant $C$ is defined in Proposition \ref{prop:as} and $\eta=1-w\underline{\alpha}\gamma/2$.
\end{theorem}

\begin{remark}
For recursion~\ref{eqn: gradient-free} we recover the linear rate  proved in \cite{kozak2019stochastic}; namely,
\begin{equation*}
	\begin{split}
		\mathbb{E}\left[f_{k}-f_*\right] & \leq \left(1-\frac{w\underline{\alpha}\gamma}{2}\right)^k \left(f_0-f_*\right).
	\end{split}
\end{equation*}
On the other hand, for  $0\leq h_k \leq \overline{h}$, the cumulative error term does not vanish:
\begin{equation}\label{error}
    \lim_{k} \ \frac{C \overline{\alpha}\overline{h}^2}{1-\eta}\left[1-\eta^k\right] =\frac{2C \overline{\alpha}\overline{h}^2}{w \underline{\alpha}\gamma} = \frac{\ell \Lambda^2 \overline{\alpha} \overline{h}^2}{4w\underline{\alpha} \gamma \min\left(1, 2-\Lambda\overline{\alpha}-w\right)},
\end{equation}
where we recall that $0<w<2-\Lambda \overline{\alpha}$ \  and $ w\leq 1$. Finally, note that for  $\alpha_k=1/\Lambda$, by Remark \ref{rem:simple} we have that the decreasing rate is $1-\gamma/(2\Lambda)$.
\end{remark}

\subsection{Sublinear rates}
We now state the results obtained with bounded step-size and vanishing discretization. Assuming only that $\seq{h_k}$ converges to zero, the objective function values converge in expectation to the optimum; while, for a polynomial decay of $h_k$, we get sublinear convergence rates. The following is an extended version of Theorem~\ref{Th:main2}  $(ii')$.
\begin{theorem}\label{th:PL_alphaconst}
		Let $\seq{x_k}$ be generated by Algorithm~\ref{eqn: derivative-free}. Assume \ref{H_lip}, \ref{H_PL}, \ref{A_Pas}, \ref{A_PE}, \ref{A_stepwith2} and \ref{A_stepbb}. If $\seq{h_k}$ converges to zero, then
	\begin{equation*}
	\begin{split}
	\lim_{k} \mathbb{E} f_{k}=f_*.
	\end{split}
	\end{equation*}
	Moreover, if $h_k=h/k^r$ for some $r>0$ and $h>0$, then
	\begin{equation*}
	    \limsup_{k} \  k^{2r}\mathbb{E}\left[f_{k}-f_*\right]\leq \frac{2C \overline{\alpha} h^2}{w \underline{\alpha} \gamma},
	\end{equation*}
    In particular, there is a constant $\tilde{C}>0$ such that
	$$\mathbb{E}\left[f_{k}-f_*\right]\leq \tilde{C}/k^{2r}$$
	and, for every $t\in\left(0,r\right)$,
	$$\mathbb{E}\left[f_{k}-f_*\right]=o\left(\frac{1}{k^{2t}}\right).$$
\end{theorem}
\begin{proof}
	 From  Lemma \ref{lem:mainineq}, we get
	\begin{equation*}
	\begin{split}
	\mathbb{E}\left[f_{k+1}-f_*\right] & \leq  \eta \mathbb{E}\left[f_{k}-f_*\right]  + C \overline{\alpha} \ h_{k}^2.
	\end{split}
	\end{equation*}
	The remainder follows from Lemma \ref{lem:aux} with $c_k=C \overline{\alpha} h^2_k$, $c=C \overline{\alpha} h^2$ and $t=2r$.
\end{proof}

In the next result we allow both the step-size and the discretization to converge to zero polynomially. In this case, we get again sublinear rates in expectation similar to the ones obtained in Theorem \ref{th:PL_alphaconst}.
\begin{theorem}\label{th:Chung}
	Let $\seq{x_k}$ be generated by Algorithm~\ref{eqn: derivative-free}. Assume \ref{H_lip}, \ref{H_PL}, \ref{A_Pas} and \ref{A_PE}. For $0<\alpha<2/\Lambda$ and $h>0$, set $\alpha_k= \alpha/k^s$, $h_k=h/k^r$ and define
	\begin{eqnarray}\label{cc}
	c  :=& w \alpha \gamma /2 \quad \text{and} \quad d:=& C \alpha h^2.
\end{eqnarray}
Then, for $s=1$ and $r>0$, we get
	\begin{equation}
\mathbb{E}\left[f_{k}-f_*\right]\leq
\begin{cases}
\frac{d}{\left(c-2r\right)k^{2r}}+o\left(\frac{1}{k^{2r}}\right) \ \ \ \ \ & \text{if} \ \ 2r<c; \\

O\left(\frac{\log k}{k^{c}} \right)\ \ \ \ \ & \text{if} \ \ 2r = c ;\\

O\left(\frac{1}{k^{c}}\right) \ \ \ \ \ & \text{if} \ \ 2r>c.
\end{cases}
\end{equation}
If $0<s<1$, for every $r>0$, we have
$$\mathbb{E}\left[f_{k}-f_*\right]\leq \frac{d}{c} \frac{1}{k^{2r}}+o\left(\frac{1}{k^{2r}}\right).$$
\end{theorem}
\begin{remark}
In order for the previous results to hold, both $\seq{\alpha_k}$ and $\seq{h_k}$ must converge to zero. In the case of $s=1$ (and so $\alpha_k$ proportional to $1/k$), as the intuition suggests, the rate improves for larger $r$ (and thus for $h_k$ vanishing faster) up to the value $c/2$. But eventually a saturation effect occurs: increasing $r$ beyond $c/2$ does not improve the bound. On the other hand, for $\alpha_k=\alpha/k^s$ with $0<s<1$, the convergence rates improve with larger values of $r$ similar to the ones in Theorem \ref{th:PL_alphaconst}.
\end{remark}
\begin{remark}
Note the difference between the results of this section. In Theorem \ref{th:PL_lin}, for non-vanishing $h_k$, we obtain a linear rate \emph{with an  error}; while in Theorems \ref{th:PL_alphaconst} and \ref{th:Chung}, with $h_k$ going to zero, the convergence rates are sublinear but \emph{to the optimum}.
\end{remark}
\begin{proof
	From Lemma \ref{lem:mainineq},
	\begin{equation*}
	\begin{split}
	\mathbb{E}\left[f_{k+1}-f_*\right] & \leq  \left(1-\frac{w\alpha_k\gamma}{2}\right) \mathbb{E}\left[f_{k}-f_*\right]  + C \alpha_k h_k^2.
	\end{split}
	\end{equation*}
    To conclude the first result, apply the first part of Lemma \ref{chung} with $p=2r$ and $c,\ d$ as in \eqref{cc}. For the second result, apply the second part of Lemma \ref{chung} with the same $c,\ d$ as before and
$
	s = s \quad \text{and} \quad t = s+2r.
$
\end{proof}
\subsection{Linear rate}
Finally, assuming a fast decay of $h_k$, we derive linear convergence rates in expectation. The following result is an extended version of Theorem~\ref{Th:main2}$(iii')$. The proof is  a simple consequence of Lemma \ref{lem:mainineq}.
\begin{theorem}\label{th:linrate}
    Let $\seq{x_k}$ be generated by Algorithm~\ref{eqn: derivative-free}. Assume \ref{H_lip}, \ref{H_PL}, \ref{A_Pas}, \ref{A_PE}, \ref{A_stepwith2} and \ref{A_stepbb}. Assume that $\seq{h_k^2/\eta_k}\in \ell^1$. Then, for every $k\in\N$,
		\begin{equation*}
	\begin{split}
		\mathbb{E}\left[f_{k}-f_*\right]
		& \leq \eta^k \left[\left(f_{0} -f_*\right)  + \frac{C\overline{\alpha}}{\eta} \sum_{j=0}^{+\infty}\frac{h_j^2}{\eta^j} \right],
	\end{split}
	\end{equation*}
	where the constant $C$ is defined in Proposition \ref{prop:as} and $\eta=1-w\underline{\alpha}\gamma/2$.

\end{theorem}
\section{Numerical results}\label{sec:num}
In this section we present synthetic examples  illustrating the different results we derived and discussed. Our analysis unifies many algorithms that have been thoroughly empirically examined, e.g., \cite{pmlr-v80-choromanski18a,berahas2019theoretical,Kozak2021Stochastic}. Hence, we present only toy problems to illustrate our theoretical results.
 We omit the convex, non-PL case as we were unable to identify a function that resulted in a qualitative difference in performance of the algorithm for any $\ell \in \{1, \ldots, d\}$.\\
{\bf Convex function satisfying PL inequality.}
Many convex functions satisfy the PL inequality leading to an improved rate of convergence. An example of this case is $f(x) = \norm{Ax}^2$ where $A \in \mathbb{R}^{n\times d}$ is fixed but not necessarily full column rank and $\lambda = 100$. Specifically, if any eigenvalue of $A$ is 0 then $f$ is not strongly convex, however because it is PL we are still able to apply Theorem \ref{Th:main2}. We choose $n=d=100$, and we force at least one eigenvalue of $A$ to be 0. For the cases $\ell < d$ we take the average of 10 runs. While in the long run the discrete gradient method catches up to the subspace approaches, it is important to recognize that for many practical problems of interest the dimension of the objective function may be very high relative to the budget for function evaluations (this budget could be due to time, money, computational power, etc). In very high-dimensional cases,  it may not even be possible to perform a single iteration of gradient descent, due to the $d+1$ function evaluations required at each iteration; requiring only $\ell$ function evaluations per iteration, may allow for substantial progress with identical budget constraints. This effect is even more apparent with the rapid initial progress made with  $\ell < d$  in the left panel of Figure \ref{fig:convexpl}.\\
\begin{figure}[t]
    \centering
    \includegraphics[width=.32\linewidth]{convex_fixed_fixed.png}
    \includegraphics[width=.32\linewidth]{convex_fixed_decaying.png}
    \includegraphics[width=.32\linewidth]{convex_sqrt_decaying.png}
    \caption{Optimization of a convex objective function satisfying the PL inequality. \textbf{left:} $\alpha = \ell/(d\lambda)$, $h=10^{-7}$. \textbf{center:} $\alpha = \ell/(d\lambda)$, $h=10^{-7}/k^{0.0001}$. \textbf{right:} $\alpha = \ell/(d\lambda\sqrt{k})$, $h=10^{-7}/k^{0.0001}$. Note the different axes between the figures.}
    \label{fig:convexpl}
\end{figure}
{\bf Non-convex function satisfying PL inequality.} $f(x) = \norm{Ax}^2 + 3\sin^2(c^\top x)$, with $A$ fixed but not necessarily full rank and $Ac = c$. Again,  let $n = d =100$ and $\lambda = 100$. For the cases $\ell < d$ we take the average of 10 runs.
\begin{figure}[t]
    \centering
    \includegraphics[width=.31\linewidth]{nonconvex_fixed_fixed.png}
    \includegraphics[width=.31\linewidth]{nonconvex_fixed_decaying.png}
    \includegraphics[width=.31\linewidth]{nonconvex_sqrt_decaying.png}
    \caption{Optimization of a non-convex objective function satisfying the PL inequality. \textbf{left:} $\alpha = \ell/(d\lambda)$, $h=10^{-7}$. \textbf{center:} $\alpha = \ell/(d\lambda)$, $h=10^{-7}/k^{0.0001}$. \textbf{right:} $\alpha = \ell/(d\lambda\sqrt{k})$, $h=10^{-7}/k^{0.0001}$. Note the different axes between the figures.}
    \label{fig:nonconvexpl}
\end{figure}
The most notable feature  in Figures \ref{fig:convexpl} and \ref{fig:nonconvexpl} is that when the step-size is not fixed, choosing $\ell < d$ severely under performs the discrete gradient method. When $\ell < d$ only a subset of the available information is being used at each iteration, and with the step-size diminishing every successive iteration has less impact than those that precede it. Thus, the trade-off between cost-per-iteration and progress-per-iteration favors a higher per-iteration cost in return for more progress, particularly in the early iterations. This trade-off flips when the step-size is fixed: much faster progress is made early on when $\ell<d$ and many directions provide improvement of the objective, but of course the discrete gradient method ultimately catches up.\\
{\bf Variability due to stochasticity}.
One potential benefit of letting $\ell=d$ and performing the full discrete gradient method is that there is no randomness involved so the results are deterministic. The theorems provide guarantees for $\mathbb{E} f(x_k)-f_*$, but here we investigate
how much variability can be expected between runs with identical initializations when $\ell < d$. We use the same non-convex function as previously, and perform 100 runs using the same initialization in each case,  considering 15000 function evaluations. The substantial overlap between the $\ell<d$ cases makes it difficult to discern between various values of $\ell$, thus in this figure we provide only the extremes, $\ell = 1$ and $\ell = d$. The bold line represents the mean of the $\ell=1$ case, and since $\ell=d$ is deterministic it is run only once.
\begin{figure}[t]
    \centering
    \includegraphics[width=.31\linewidth]{convexSimulations_fixed.png}
    \includegraphics[width=.31\linewidth]{convexSimulations_fixed_decaying.png}
    \includegraphics[width=.31\linewidth]{convexSimulations_sqrt_decaying.png}
    \caption{Optimization of a convex objective function with 100 restarts at the same initialization.  \textbf{left:} $\alpha = \ell/(d\lambda)$, $h=10^{-7}$. \textbf{center:} $\alpha = \ell/(d\lambda)$, $h=10^{-7}/k^{.0001}$. \textbf{right:} $\alpha = \ell/(d\lambda\sqrt{k})$, $h=10^{-7}/k^{1.0001}$. Note the different axes between the figures.}
    \label{fig:simulations}
\end{figure}
As expected, there is variability between runs when $\ell < d$, however in the early iterations even the worst case performs better than discretized gradient descent. We have only a heuristic explanation for this phenomenon: in the early iterations there are many directions that may lead to improvement so the inexpensive $\ell=1$ directions are more efficient than a full gradient estimate, but as we approach the optimum a judicious choice of direction is rewarded. The fact that the discrete gradient method catches up and eventually outperforms the others is consistent with the theory since
all else equal, larger $\ell$ implies better progress per iteration. Of greater practical interest is the out-performance in the scenario $\ell \ll d$ in the early iterations. This suggests that for low-precision optimization, or when relatively few iterations are possible due to time or money constraints, it may be beneficial to choose $\ell <d$. Indeed, this is precisely the scenario where subspace descent methods are used, cf. \cite{Kozak2021Stochastic,pmlr-v80-choromanski18a,NIPS2018_7451}. Our theory does not cover the use of a backtracking line search such as the one presented in \cite{berahas2019global}, so we do not provide figures detailing its performance; however, in practice a line search is a necessary component to achieve outperformance compared to the gradient method as discussed at length in \cite{Kozak2021Stochastic}.\\
{\bf Effect of $h$.} Recall that there is an additive error term at each iteration due to the use of finite differences to approximate the gradient. In light of this fact, it may be surprising that the preceding figures appear to have objective function values that decrease monotonically with the increase in function evaluations even when $h$ is fixed. Note, however, that for a fixed $h$ as in Theorem \ref{th:PL_lin} the  error is asymptotically $\mathcal{O}(h^2)$, whereas in Theorem \ref{th:PL_alphaconst} a decaying $h_k = h/k^r$ is used with $r, h >0$, resulting in an objective that decays to zero.
Figure \ref{fig:h_examples} demonstrates that the limiting error, or lack thereof, guaranteed by the theorems is observed in practice. The first figure is a convex function satisfying the PL inequality with various fixed values for $h$.  The second is a non-convex function satisfying the PL inequality with the same values of $h$, and in the third figure we set $h_k = h/k^r$ with $h=10^{-5}$ and $r = 1$ and run the algorithm 100 times on the non-convex function. In all cases, $d = 5$, $\ell = 1$ and $\lambda = 4$, resulting in fast convergence. \\
Several conclusions can be drawn from Figure \ref{fig:h_examples}. First, as expected by the theorems, $h$ does not appear to play a role in the rate of convergence, only in the magnitude of the asymptotic error.
Further, for all of the algorithms that fit our theory the finite difference error can essentially be ignored,  provided the desired accuracy is less than $\mathcal{O}(h^2)$. So, with $h$ the square root of machine precision, our theorems seem to indicate (and our figures support) that the error due to finite differences can be safely ignored. The figure on the right shows that the variance due to different stochastic realizations of the algorithm is small, suggesting that the theoretical analysis done in \cite{Kozak2021Stochastic} and \cite{kozak2019stochastic} may be extended to the discrete setting.

\begin{figure}[ht]
    \centering
    \includegraphics[width=.31\linewidth]{convexPL_h.png}
    \includegraphics[width=.31\linewidth]{nonconvexPL_h.png}
    \includegraphics[width=.31\linewidth]{nonconvexPL_hdecreasing.png}

    \caption{\textbf{left:} Optimization of a convex PL function using various values of $h$. \textbf{center:} Optimization of a non-convex PL function using various values of $h$. \textbf{right:} Optimization of a non-convex PL function with $h_k = 10^{-5}/k$ }
    \label{fig:h_examples}
\end{figure}

\section{Conclusions}\label{sec:conc}
We presented a method that generalizes several well-known derivative-free optimization algorithms including, for example, spherical smoothing and discretized versions of coordinate and gradient descent.
We provide  convergence analysis of this generic method
considering objective functions that are either convex or satisfy a Polyak-Łojasiewicz (PL) condition. Multiple possible choices for the stepsizes and the finite difference  parameter are studied.  The best choice depends on the error tolerance of the user. By allowing the stepsize and discretization to decay the algorithm achieves a slower rate of convergence but is able to converge to an optimum, but by fixing these values a faster convergence rate is obtained at the expense of converging only to within a region of an optimum.
To the best of our knowledge, this work provides the first convergence guarantees for the iterates of spherical smoothing and discretized coordinate descent to a minimizer when the objective function is convex.

There are several possible extensions of this work.
Our analysis does not cover the use of an adaptive stepsize that has been shown empirically to be highly advantageous  \cite{NIPS2018_7451,Kozak2021Stochastic}. Theoretical analysis using a stochastic linesearch may be possible, several recent papers offer promising results that may extend to our case \cite{cartis2018global,berahas2019global,MR4060460}. It would be interesting to analyze
the algorithm in the presence of noisy function evaluations as in \cite{MR50243,MR499560}. Such an extension would be of great practical consequence as many applications of interest have noisy objective functions. Finally, faster convergence may be possible using derivative-free quasi-Newton methods or any methods that exploit the curvature of the objective as in \cite{MR3936038,pmlr-v119-hanzely20a,pmlr-v119-hanzely20b,bollapragada2019adaptive}.

\bibliographystyle{siam}