  Deep Neural Network (DNN) based inference at the edge is challenging as these
compute and data-intensive algorithms need to be implemented at low cost and
low power while meeting the latency constraints of the target applications.
Sparsity, in both activations and weights inherent to DNNs, is a key knob to
leverage. In this paper, we present RAMAN, a Re-configurable and spArse tinyML
Accelerator for infereNce on edge, architected to exploit the sparsity to
reduce area (storage), power as well as latency. RAMAN can be configured to
support a wide range of DNN topologies - consisting of different convolution
layer types and a range of layer parameters (feature-map size and the number of
channels). RAMAN can also be configured to support accuracy vs power/latency
tradeoffs using techniques deployed at compile-time and run-time. We present
the salient features of the architecture, provide implementation results and
compare the same with the state-of-the-art. RAMAN employs novel dataflow
inspired by Gustavson's algorithm that has optimal input activation (IA) and
output activation (OA) reuse to minimize memory access and the overall data
movement cost. The dataflow allows RAMAN to locally reduce the partial sum
(Psum) within a processing element array to eliminate the Psum writeback
traffic. Additionally, we suggest a method to reduce peak activation memory by
overlapping IA and OA on the same memory space, which can reduce storage
requirements by up to 50%. RAMAN was implemented on a low-power and
resource-constrained Efinix Ti60 FPGA with 37.2K LUTs and 8.6K register
utilization. RAMAN processes all layers of the MobileNetV1 model at 98.47
GOp/s/W and the DS-CNN model at 79.68 GOp/s/W by leveraging both weight and
activation sparsity.
