  Cutting-edge embedded system applications, such as self-driving cars and
unmanned drone software, are reliant on integrated CPU/GPU platforms for their
DNNs-driven workload, such as perception and other highly parallel components.
In this work, we set out to explore the hidden performance implication of GPU
memory management methods of integrated CPU/GPU architecture. Through a series
of experiments on micro-benchmarks and real-world workloads, we find that the
performance under different memory management methods may vary according to
application characteristics. Based on this observation, we develop a
performance model that can predict system overhead for each memory management
method based on application characteristics. Guided by the performance model,
we further propose a runtime scheduler. By conducting per-task memory
management policy switching and kernel overlapping, the scheduler can
significantly relieve the system memory pressure and reduce the multitasking
co-run response time. We have implemented and extensively evaluated our system
prototype on the NVIDIA Jetson TX2, Drive PX2, and Xavier AGX platforms, using
both Rodinia benchmark suite and two real-world case studies of drone software
and autonomous driving software.
