  We consider Markov chains that obey the following general non-linear state
space model: $\Phi_{k+1} = F(\Phi_k, \alpha(\Phi_k, U_{k+1}))$ where the
function $F$ is $C^1$ while $\alpha$ is typically discontinuous and $\{U_k: k
\in \mathbb{Z}_{> 0} \}$ is an independent and identically distributed process.
We assume that for all $x$, the random variable $\alpha(x, U_1)$ admits a
density $p_x$ such that $(x, w) \mapsto p_x(w)$ is lower semi-continuous.
  We generalize and extend previous results that connect properties of the
underlying deterministic control model to provide conditions for the chain to
be $\varphi$-irreducible and aperiodic. By building on those results, we show
that if a rank condition on the controllability matrix is satisfied for all
$x$, there is equivalence between the existence of a globally attracting state
for the control model and $\varphi$-irreducibility of the Markov chain.
Additionally, under the same rank condition on the controllability matrix, we
prove that there is equivalence between the existence of a steadily attracting
state and the $\varphi$-irreducibility and aperiodicity of the chain. The
notion of steadily attracting state is new.
  Those results hold under considerably weaker assumptions on the model than
previous ones that would require $(x,u) \mapsto F(x,\alpha(x,u))$ to be
$C^\infty$ (while it can be discontinuous here). Additionally the establishment
of a necessary and sufficient condition for the $\varphi$-irreducibility and
aperiodicity without a structural assumption on the control set is novel---even
for Markov chains where $(x,u) \mapsto F(x,\alpha(x,u))$ is $C^\infty$.
  We illustrate that the conditions are easy to verify on a non-trivial and
non-artificial example of Markov chain arising in the context of adaptive
stochastic search algorithms to optimize continuous functions in a black-box
scenario.
