  The effectiveness of Recurrent Neural Networks (RNNs) for tasks such as
Automatic Speech Recognition has fostered interest in RNN inference
acceleration. Due to the recurrent nature and data dependencies of RNN
computations, prior work has designed customized architectures specifically
tailored to the computation pattern of RNN, getting high computation efficiency
for certain chosen model sizes. However, given that the dimensionality of RNNs
varies a lot for different tasks, it is crucial to generalize this efficiency
to diverse configurations. In this work, we identify adaptiveness as a key
feature that is missing from today's RNN accelerators. In particular, we first
show the problem of low resource-utilization and low adaptiveness for the
state-of-the-art RNN implementations on GPU, FPGA and ASIC architectures. To
solve these issues, we propose an intelligent tiled-based dispatching mechanism
for increasing the adaptiveness of RNN computation, in order to efficiently
handle the data dependencies. To do so, we propose Sharp as a hardware
accelerator, which pipelines RNN computation using an effective scheduling
scheme to hide most of the dependent serialization. Furthermore, Sharp employs
dynamic reconfigurable architecture to adapt to the model's characteristics.
Sharp achieves 2x, 2.8x, and 82x speedups on average, considering different RNN
models and resource budgets, compared to the state-of-the-art ASIC, FPGA, and
GPU implementations, respectively. Furthermore, we provide significant
energy-reduction with respect to the previous solutions, due to the low power
dissipation of Sharp (321 GFLOPS/Watt).
