Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Cocktail: Leveraging Ensemble Learning for Optimized Model Serving in
  Public Cloud   With a growing demand for adopting ML models for a varietyof application
services, it is vital that the frameworks servingthese models are capable of
delivering highly accurate predic-tions with minimal latency along with reduced
deploymentcosts in a public cloud environment. Despite high latency,prior works
in this domain are crucially limited by the accu-racy offered by individual
models. Intuitively, model ensem-bling can address the accuracy gap by
intelligently combiningdifferent models in parallel. However, selecting the
appro-priate models dynamically at runtime to meet the desiredaccuracy with low
latency at minimal deployment cost is anontrivial problem. Towards this, we
proposeCocktail, a costeffective ensembling-based model serving
framework.Cock-tailcomprises of two key components: (i) a dynamic
modelselection framework, which reduces the number of modelsin the ensemble,
while satisfying the accuracy and latencyrequirements; (ii) an adaptive
resource management (RM)framework that employs a distributed proactive
autoscalingpolicy combined with importance sampling, to efficiently allo-cate
resources for the models. The RM framework leveragestransient virtual machine
(VM) instances to reduce the de-ployment cost in a public cloud. A prototype
implementationofCocktailon the AWS EC2 platform and exhaustive evalua-tions
using a variety of workloads demonstrate thatCocktailcan reduce deployment cost
by 1.45x, while providing 2xreduction in latency and satisfying the target
accuracy for upto 96% of the requests, when compared to
state-of-the-artmodel-serving frameworks.

33
The categories of this paper are: plasm-ph
23
The categories of this paper are: solv-int
21
The categories of this paper are: adap-org
18
The categories of this paper are: econ
12
The categories of this paper are: nucl-th
7
The categories of this paper are: hep-th
6
The categories of this paper are: astro-ph
17
The categories of this paper are: q-fin
11
The categories of this paper are: hep-lat
36
The categories of this paper are: cmp-lg
37
The categories of this paper are: supr-con
16
The categories of this paper are: stat
31
The categories of this paper are: ao-sci
20
The categories of this paper are: acc-phys
8
The categories of this paper are: hep-ex
10
The categories of this paper are: q-bio