Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Salus: Fine-Grained GPU Sharing Primitives for Deep Learning
  Applications   GPU computing is becoming increasingly more popular with the proliferation of
deep learning (DL) applications. However, unlike traditional resources such as
CPU or the network, modern GPUs do not natively support fine-grained sharing
primitives. Consequently, implementing common policies such as time sharing and
preemption are expensive. Worse, when a DL application cannot completely use a
GPU's resources, the GPU cannot be efficiently shared between multiple
applications, leading to GPU underutilization.
  We present Salus to enable two GPU sharing primitives: fast job switching and
memory sharing, in order to achieve fine-grained GPU sharing among multiple DL
applications. Salus implements an efficient, consolidated execution service
that exposes the GPU to different DL applications, and enforces fine-grained
sharing by performing iteration scheduling and addressing associated memory
management issues. We show that these primitives can then be used to implement
flexible sharing policies such as fairness, prioritization, and packing for
various use cases. Our integration of Salus with TensorFlow and evaluation on
popular DL jobs show that Salus can improve the average completion time of DL
training jobs by $3.19\times$, GPU utilization for hyper-parameter tuning by
$2.38\times$, and GPU utilization of DL inference applications by $42\times$
over not sharing the GPU and $7\times$ over NVIDIA MPS with small overhead.

4
The categories of this paper are: cond-mat
31
The categories of this paper are: ao-sci
2
The categories of this paper are: math
19
The categories of this paper are: eess
36
The categories of this paper are: cmp-lg
6
The categories of this paper are: astro-ph
23
The categories of this paper are: solv-int
10
The categories of this paper are: q-bio
32
The categories of this paper are: atom-ph
22
The categories of this paper are: chao-dyn
17
The categories of this paper are: q-fin
16
The categories of this paper are: stat
28
The categories of this paper are: alg-geom
15
The categories of this paper are: nucl-ex
24
The categories of this paper are: dg-ga
14
The categories of this paper are: math-ph