  Dynamic graph neural network (DGNN) is becoming increasingly popular because
of its widespread use in capturing dynamic features in the real world. A
variety of dynamic graph neural networks designed from algorithmic perspectives
have succeeded in incorporating temporal information into graph processing.
Despite the promising algorithmic performance, deploying DGNNs on hardware
presents additional challenges due to the model complexity, diversity, and the
nature of the time dependency. Meanwhile, the differences between DGNNs and
static graph neural networks make hardware-related optimizations for static
graph neural networks unsuitable for DGNNs. In this paper, we select eight
prevailing DGNNs with different characteristics and profile them on both CPU
and GPU. The profiling results are summarized and analyzed, providing in-depth
insights into the bottlenecks of DGNNs on hardware and identifying potential
optimization opportunities for future DGNN acceleration. Followed by a
comprehensive survey, we provide a detailed analysis of DGNN performance
bottlenecks on hardware, including temporal data dependency, workload
imbalance, data movement, and GPU warm-up. We suggest several optimizations
from both software and hardware perspectives. This paper is the first to
provide an in-depth analysis of the hardware performance of DGNN Code is
available at https://github.com/sharc-lab/DGNN_analysis.
