Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Runtime Deep Model Multiplexing for Reduced Latency and Energy
  Consumption Inference   We propose a learning algorithm to design a light-weight neural multiplexer
that given the input and computational resource requirements, calls the model
that will consume the minimum compute resources for a successful inference.
Mobile devices can use the proposed algorithm to offload the hard inputs to the
cloud while inferring the easy ones locally. Besides, in the large scale
cloud-based intelligent applications, instead of replicating the most-accurate
model, a range of small and large models can be multiplexed from depending on
the input's complexity which will save the cloud's computational resources. The
input complexity or hardness is determined by the number of models that can
predict the correct label. For example, if no model can predict the label
correctly, then the input is considered as the hardest. The proposed algorithm
allows the mobile device to detect the inputs that can be processed locally and
the ones that require a larger model and should be sent a cloud server.
Therefore, the mobile user benefits from not only the local processing but also
from an accurate model hosted on a cloud server. Our experimental results show
that the proposed algorithm improves mobile's model accuracy by 8.52% which is
because of those inputs that are properly selected and offloaded to the cloud
server. In addition, it saves the cloud providers' compute resources by a
factor of 2.85x as small models are chosen for easier inputs.

11
The categories of this paper are: hep-lat
23
The categories of this paper are: solv-int
36
The categories of this paper are: cmp-lg
3
The categories of this paper are: physics
16
The categories of this paper are: stat
37
The categories of this paper are: supr-con
18
The categories of this paper are: econ
17
The categories of this paper are: q-fin
30
The categories of this paper are: q-alg
9
The categories of this paper are: nlin
28
The categories of this paper are: alg-geom
20
The categories of this paper are: acc-phys
13
The categories of this paper are: quant-ph
32
The categories of this paper are: atom-ph
7
The categories of this paper are: hep-th
8
The categories of this paper are: hep-ex