  Biological neural networks are equipped with an inherent capability to
continuously adapt through online learning. This aspect remains in stark
contrast to learning with error backpropagation through time (BPTT) applied to
recurrent neural networks (RNNs), or recently to biologically-inspired spiking
neural networks (SNNs). BPTT involves offline computation of the gradients due
to the requirement to unroll the network through time. Online learning has
recently regained the attention of the research community, focusing either on
approaches that approximate BPTT or on biologically-plausible schemes applied
to SNNs. Here we present an alternative perspective that is based on a clear
separation of spatial and temporal gradient components. Combined with insights
from biology, we derive from first principles a novel online learning algorithm
for deep SNNs, called online spatio-temporal learning (OSTL). For shallow
networks, OSTL is gradient-equivalent to BPTT enabling for the first time
online training of SNNs with BPTT-equivalent gradients. In addition, the
proposed formulation unveils a class of SNN architectures trainable online at
low time complexity. Moreover, we extend OSTL to a generic form, applicable to
a wide range of network architectures, including networks comprising long
short-term memory (LSTM) and gated recurrent units (GRU). We demonstrate the
operation of our algorithm on various tasks from language modelling to speech
recognition and obtain results on par with the BPTT baselines. The proposed
algorithm provides a framework for developing succinct and efficient online
training approaches for SNNs and in general deep RNNs.
