  Latency-critical services have been widely deployed in cloud environments.
For cost-efficiency, multiple services are usually co-located on a server.
Thus, run-time resource scheduling becomes the pivot for QoS control in these
complicated co-location cases. However, the scheduling exploration space
enlarges rapidly with the increasing server resources, making the schedulers
hardly provide ideal solutions quickly. More importantly, we observe that there
are "resource cliffs" in the scheduling exploration space. They affect the
exploration efficiency and always lead to severe QoS fluctuations. Resource
cliffs cannot be easily avoided in previous schedulers. To address these
problems, we propose a novel ML-based intelligent scheduler - OSML. It learns
the correlation between architectural hints (e.g., IPC, cache misses, memory
footprint, etc.), scheduling solutions and the QoS demands based on a data set
we collected from 11 widely deployed services running on off-the-shelf servers.
OSML employs multiple ML models to work collaboratively to predict QoS
variations, shepherd the scheduling, and recover from QoS violations in
complicated co-location cases. OSML can intelligently avoid resource cliffs
during scheduling and reach an optimal solution much faster than previous
approaches for co-located LC services. Experimental results show that OSML
supports higher loads and meets QoS targets with lower scheduling overheads and
shorter convergence time than previous studies.
