  In this paper, we elaborate over the well-known interpretability issue in
echo state networks. The idea is to investigate the dynamics of reservoir
neurons with time-series analysis techniques taken from research on complex
systems. Notably, we analyze time-series of neuron activations with Recurrence
Plots (RPs) and Recurrence Quantification Analysis (RQA), which permit to
visualize and characterize high-dimensional dynamical systems. We show that
this approach is useful in a number of ways. First, the two-dimensional
representation offered by RPs provides a way for visualizing the
high-dimensional dynamics of a reservoir. Our results suggest that, if the
network is stable, reservoir and input denote similar line patterns in the
respective RPs. Conversely, the more unstable the ESN, the more the RP of the
reservoir presents instability patterns. As a second result, we show that the
$\mathrm{L_{max}}$ measure is highly correlated with the well-established
maximal local Lyapunov exponent. This suggests that complexity measures based
on RP diagonal lines distribution provide a valuable tool to quantify the
degree of network stability. Finally, our analysis shows that all RQA measures
fluctuate on the proximity of the so-called edge of stability, where an ESN
typically achieves maximum computational capability. We verify that the
determination of the edge of stability provided by such RQA measures is more
accurate than two well-known criteria based on the Jacobian matrix of the
reservoir. Therefore, we claim that RPs and RQA-based analyses can be used as
valuable tools to design an effective network given a specific problem.
