  Skeleton-based Graph Convolutional Networks (GCNs) models for action
recognition have achieved excellent prediction accuracy in the field. However,
limited by large model and computation complexity, GCNs for action recognition
like 2s-AGCN have insufficient power-efficiency and throughput on GPU. Thus,
the demand of model reduction and hardware acceleration for low-power GCNs
action recognition application becomes continuously higher.
  To address challenges above, this paper proposes a runtime sparse feature
compress accelerator with hybrid pruning method: RFC-HyPGCN. First, this method
skips both graph and spatial convolution workloads by reorganizing the
multiplication order. Following spatial convolution workloads channel-pruning
dataflow, a coarse-grained pruning method on temporal filters is designed,
together with sampling-like fine-grained pruning on time dimension. Later, we
come up with an architecture where all convolutional layers are mapped on chip
to pursue high throughput. To further reduce storage resource utilization,
online sparse feature compress format is put forward. Features are divided and
encoded into several banks according to presented format, then bank storage is
split into depth-variable mini-banks. Furthermore, this work applies
quantization, input-skipping and intra-PE dynamic data scheduling to accelerate
the model. In experiments, proposed pruning method is conducted on 2s-AGCN,
acquiring 3.0x-8.4x model compression ratio and 73.20\% graph-skipping
efficiency with balancing weight pruning. Implemented on Xilinx XCKU-115 FPGA,
the proposed architecture has the peak performance of 1142 GOP/s and achieves
up to 9.19x and 3.91x speedup over high-end GPU NVIDIA 2080Ti and NVIDIA V100,
respectively. Compared with latest accelerator for action recognition GCNs
models, our design reaches 22.9x speedup and 28.93\% improvement on DSP
efficiency.
