  Due to limited resources on edge and different characteristics of deep neural
network (DNN) models, it is a big challenge to optimize DNN inference
performance in terms of energy consumption and end-to-end latency on edge
devices. In addition to the dynamic voltage frequency scaling (DVFS) technique,
the edge-cloud architecture provides a collaborative approach for efficient DNN
inference. However, current edge-cloud collaborative inference methods have not
optimized various compute resources on edge devices. Thus, we propose DVFO, a
novel DVFS-enabled edge-cloud collaborative inference framework, which
co-optimizes DVFS and offloading parameters via deep reinforcement learning
(DRL). Specifically, DVFO automatically co-optimizes 1) the CPU, GPU and memory
frequencies of edge devices, and 2) the feature maps to be offloaded to cloud
servers. In addition, it leverages a thinking-while-moving concurrent mechanism
to accelerate the DRL learning process, and a spatial-channel attention
mechanism to extract DNN feature maps of secondary importance for workload
offloading. This approach improves inference performance for different DNN
models under various edge-cloud network conditions. Extensive evaluations using
two datasets and six widely-deployed DNN models on three heterogeneous edge
devices show that DVFO significantly reduces the energy consumption by 33% on
average, compared to state-of-the-art schemes. Moreover, DVFO achieves up to
28.6%-59.1% end-to-end latency reduction, while maintaining accuracy within 1%
loss on average.
