  Deep neural networks (DNNs) have substantial computational and memory
requirements, and the compilation of its computational graphs has a great
impact on the performance of resource-constrained (e.g., computation, I/O, and
memory-bound) edge computing systems. While efficient execution of their
computational graph requires an effective scheduling algorithm, generating the
optimal scheduling solution is a challenging NP-hard problem. Furthermore, the
complexity of scheduling DNN computational graphs will further increase on
pipelined multi-core systems considering memory communication cost, as well as
the increasing size of DNNs. Using the synthetic graph for the training
dataset, this work presents a reinforcement learning (RL) based scheduling
framework RESPECT, which learns the behaviors of optimal optimization
algorithms and generates near-optimal scheduling results with short solving
runtime overhead. Our framework has demonstrated up to $\sim2.5\times$
real-world on-chip inference runtime speedups over the commercial compiler with
ten popular ImageNet models deployed on the physical Coral Edge TPUs system.
Moreover, compared to the exact optimization methods, the proposed RL
scheduling improves the scheduling optimization runtime by up to 683$\times$
speedups compared to the commercial compiler and matches the exact optimal
solutions with up to 930$\times$ speedups. Finally, we perform a comprehensive
generalizability test, which demonstrates RESPECT successfully imitates optimal
solving behaviors from small synthetic graphs to large real-world DNNs
computational graphs.
