  It is a widely accepted fact that data representations intervene noticeably
in machine learning tools. The more they are well defined the better the
performance results are. Feature extraction-based methods such as autoencoders
are conceived for finding more accurate data representations from the original
ones. They efficiently perform on a specific task in terms of 1) high accuracy,
2) large short term memory and 3) low execution time. Echo State Network (ESN)
is a recent specific kind of Recurrent Neural Network which presents very rich
dynamics thanks to its reservoir-based hidden layer. It is widely used in
dealing with complex non-linear problems and it has outperformed classical
approaches in a number of tasks including regression, classification, etc. In
this paper, the noticeable dynamism and the large memory provided by ESN and
the strength of Autoencoders in feature extraction are gathered within an ESN
Recurrent Autoencoder (ESN-RAE). In order to bring up sturdier alternative to
conventional reservoir-based networks, not only single layer basic ESN is used
as an autoencoder, but also Multi-Layer ESN (ML-ESN-RAE). The new features,
once extracted from ESN's hidden layer, are applied to classification tasks.
The classification rates rise considerably compared to those obtained when
applying the original data features. An accuracy-based comparison is performed
between the proposed recurrent AEs and two variants of an ELM feed-forward AEs
(Basic and ML) in both of noise free and noisy environments. The empirical
study reveals the main contribution of recurrent connections in improving the
classification performance results.
