  Recurrent neural networks (RNNs) are complex dynamical systems, capable of
ongoing activity without any driving input. The long-term behavior of
free-running RNNs, described by periodic, chaotic and fixed point attractors,
is controlled by the statistics of the neural connection weights, such as the
density $d$ of non-zero connections, or the balance $b$ between excitatory and
inhibitory connections. However, for information processing purposes, RNNs need
to receive external input signals, and it is not clear which of the dynamical
regimes is optimal for this information import. We use both the average
correlations $C$ and the mutual information $I$ between the momentary input
vector and the next system state vector as quantitative measures of information
import and analyze their dependence on the balance and density of the network.
Remarkably, both resulting phase diagrams $C(b,d)$ and $I(b,d)$ are highly
consistent, pointing to a link between the dynamical systems and the
information-processing approach to complex systems. Information import is
maximal not at the 'edge of chaos', which is optimally suited for computation,
but surprisingly in the low-density chaotic regime and at the border between
the chaotic and fixed point regime. Moreover, we find a completely new type of
resonance phenomenon, called 'Import Resonance' (IR), where the information
import shows a maximum, i.e. a peak-like dependence on the coupling strength
between the RNN and its input. IR complements Recurrence Resonance (RR), where
correlation and mutual information of successive system states peak for a
certain amplitude of noise added to the system. Both IR and RR can be exploited
to optimize information processing in artificial neural networks and might also
play a crucial role in biological neural systems.
