  Dataflow architectures are growing in popularity due to their potential to
mitigate the challenges posed by the memory wall inherent to the Von Neumann
architecture. At the same time, high-level synthesis (HLS) has demonstrated its
efficacy as a design methodology for generating efficient dataflow
architectures within a short development cycle. However, existing HLS tools
rely on developers to explore the vast dataflow design space, ultimately
leading to suboptimal designs. This phenomenon is especially concerning as the
size of the HLS design grows. To tackle these challenges, we introduce HIDA, a
new scalable and hierarchical HLS framework that can systematically convert an
algorithmic description into a dataflow implementation on hardware. We first
propose a collection of efficient and versatile dataflow representations for
modeling the hierarchical dataflow structure. Capitalizing on these
representations, we develop an automated optimizer that decomposes the dataflow
optimization problem into multiple levels based on the inherent dataflow
hierarchy. Using FPGAs as an evaluation platform, working with a set of neural
networks modeled in PyTorch, HIDA achieves up to 8.54$\times$ higher throughput
compared to the state-of-the-art (SOTA) HLS optimization tool. Furthermore,
despite being fully automated and able to handle various applications, HIDA
achieves 1.29$\times$ higher throughput over the SOTA RTL-based neural network
accelerators on an FPGA.
