  Deep learning needs high-precision handling of forwarding signals,
backpropagating errors, and updating weights. This is inherently required by
the learning algorithm since the gradient descent learning rule relies on the
chain product of partial derivatives. However, it is challenging to implement
deep learning in hardware systems that use noisy analog memristors as
artificial synapses, as well as not being biologically plausible.
Memristor-based implementations generally result in an excessive cost of
neuronal circuits and stringent demands for idealized synaptic devices. Here,
we demonstrate that the requirement for high precision is not necessary and
that more efficient deep learning can be achieved when this requirement is
lifted. We propose a binary stochastic learning algorithm that modifies all
elementary neural network operations, by introducing (i) stochastic
binarization of both the forwarding signals and the activation function
derivatives, (ii) signed binarization of the backpropagating errors, and (iii)
step-wised weight updates. Through an extensive hybrid approach of software
simulation and hardware experiments, we find that binary stochastic deep
learning systems can provide better performance than the software-based
benchmarks using the high-precision learning algorithm. Also, the binary
stochastic algorithm strongly simplifies the neural network operations in
hardware, resulting in an improvement of the energy efficiency for the
multiply-and-accumulate operations by more than three orders of magnitudes.
