Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Collage Inference: Using Coded Redundancy for Low Variance Distributed
  Image Classification   MLaaS (ML-as-a-Service) offerings by cloud computing platforms are becoming
increasingly popular. Hosting pre-trained machine learning models in the cloud
enables elastic scalability as the demand grows. But providing low latency and
reducing the latency variance is a key requirement. Variance is harder to
control in a cloud deployment due to uncertainties in resource allocations
across many virtual instances. We propose the collage inference technique which
uses a novel convolutional neural network model, collage-cnn, to provide
low-cost redundancy. A collage-cnn model takes a collage image formed by
combining multiple images and performs multi-image classification in one shot,
albeit at slightly lower accuracy. We augment a collection of traditional
single image classifier models with a single collage-cnn classifier which acts
as their low-cost redundant backup. Collage-cnn provides backup classification
results if any single image classification requests experience slowdown.
Deploying the collage-cnn models in the cloud, we demonstrate that the 99th
percentile tail latency of inference can be reduced by 1.2x to 2x compared to
replication based approaches while providing high accuracy. Variation in
inference latency can be reduced by 1.8x to 15x.

5
The categories of this paper are: gr-qc
32
The categories of this paper are: atom-ph
37
The categories of this paper are: supr-con
6
The categories of this paper are: astro-ph
13
The categories of this paper are: quant-ph
26
The categories of this paper are: bayes-an
25
The categories of this paper are: patt-sol
11
The categories of this paper are: hep-lat
8
The categories of this paper are: hep-ex
12
The categories of this paper are: nucl-th
24
The categories of this paper are: dg-ga
18
The categories of this paper are: econ
30
The categories of this paper are: q-alg
20
The categories of this paper are: acc-phys
15
The categories of this paper are: nucl-ex
3
The categories of this paper are: physics