  Tensor computations overwhelm traditional general-purpose computing devices
due to the large amounts of data and operations of the computations. They call
for a holistic solution composed of both hardware acceleration and software
mapping. Hardware/software (HW/SW) co-design optimizes the hardware and
software in concert and produces high-quality solutions. There are two main
challenges in the co-design flow. First, multiple methods exist to partition
tensor computation and have different impacts on performance and energy
efficiency. Besides, the hardware part must be implemented by the intrinsic
functions of spatial accelerators. It is hard for programmers to identify and
analyze the partitioning methods manually. Second, the overall design space
composed of HW/SW partitioning, hardware optimization, and software
optimization is huge. The design space needs to be efficiently explored.
  To this end, we propose an agile co-design approach HASCO that provides an
efficient HW/SW solution to dense tensor computation. We use tensor syntax
trees as the unified IR, based on which we develop a two-step approach to
identify partitioning methods. For each method, HASCO explores the hardware and
software design spaces. We propose different algorithms for the explorations,
as they have distinct objectives and evaluation costs. Concretely, we develop a
multi-objective Bayesian optimization algorithm to explore hardware
optimization. For software optimization, we use heuristic and Q-learning
algorithms. Experiments demonstrate that HASCO achieves a 1.25X to 1.44X
latency reduction through HW/SW co-design compared with developing the hardware
and software separately.
