  The need for deep neural network (DNN) models with higher performance and
better functionality leads to the proliferation of very large models. Model
training, however, requires intensive computation time and energy.
Memristor-based compute-in-memory (CIM) modules can perform vector-matrix
multiplication (VMM) in situ and in parallel, and have shown great promises in
DNN inference applications. However, CIM-based model training faces challenges
due to non-linear weight updates, device variations, and low-precision in
analog computing circuits. In this work, we experimentally implement a
mixed-precision training scheme to mitigate these effects using a
bulk-switching memristor CIM module. Lowprecision CIM modules are used to
accelerate the expensive VMM operations, with high precision weight updates
accumulated in digital units. Memristor devices are only changed when the
accumulated weight update value exceeds a pre-defined threshold. The proposed
scheme is implemented with a system-on-chip (SoC) of fully integrated analog
CIM modules and digital sub-systems, showing fast convergence of LeNet training
to 97.73%. The efficacy of training larger models is evaluated using realistic
hardware parameters and shows that that analog CIM modules can enable efficient
mix-precision DNN training with accuracy comparable to full-precision software
trained models. Additionally, models trained on chip are inherently robust to
hardware variations, allowing direct mapping to CIM inference chips without
additional re-training.
