  Many hardware vendors have introduced specialized deep neural networks (DNN)
accelerators owing to their superior performance and efficiency. As such, how
to generate and optimize the code for the hardware accelerator becomes an
important yet less explored problem. In this paper, we perform the
compiler-stage optimization study using a novel and representative Cambricon
DNN accelerator and demonstrate that the code optimization knobs play an
important role in unleashing the potential of hardware computational
horsepower. However, even only two studied code optimization knobs, namely the
number of cores and layer fusion scheme, present an enormous search space that
prevents the naive brute-force search. This work introduces a joint,
auto-tuning optimization framework to address this challenge. We first use a
set of synthesized DNN layers to study the interplay between the hardware
performance and layer characteristics. Based on the insights, we extract the
operation count and feature map channel size as each layer's characteristics
and derive a joint optimization strategy to decide the performance-optimal core
number and fusion scheme. We evaluate the performance of the proposed approach
using a set of representative DNN models and show that it achieves the minimal
of 3.6x and the maximal of 7.9x performance speedup compared to no optimization
baseline. We also show that the achieved speedup is close to the oracle case
that is based on a reduced brute-force search but with much less search time.
