Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Private Model Compression via Knowledge Distillation   The soaring demand for intelligent mobile applications calls for deploying
powerful deep neural networks (DNNs) on mobile devices. However, the
outstanding performance of DNNs notoriously relies on increasingly complex
models, which in turn is associated with an increase in computational expense
far surpassing mobile devices' capacity. What is worse, app service providers
need to collect and utilize a large volume of users' data, which contain
sensitive information, to build the sophisticated DNN models. Directly
deploying these models on public mobile devices presents prohibitive privacy
risk. To benefit from the on-device deep learning without the capacity and
privacy concerns, we design a private model compression framework RONA.
Following the knowledge distillation paradigm, we jointly use hint learning,
distillation learning, and self learning to train a compact and fast neural
network. The knowledge distilled from the cumbersome model is adaptively
bounded and carefully perturbed to enforce differential privacy. We further
propose an elegant query sample selection method to reduce the number of
queries and control the privacy loss. A series of empirical evaluations as well
as the implementation on an Android mobile device show that RONA can not only
compress cumbersome models efficiently but also provide a strong privacy
guarantee. For example, on SVHN, when a meaningful
$(9.83,10^{-6})$-differential privacy is guaranteed, the compact model trained
by RONA can obtain 20$\times$ compression ratio and 19$\times$ speed-up with
merely 0.97% accuracy loss.

13
The categories of this paper are: quant-ph
35
The categories of this paper are: mtrl-th
6
The categories of this paper are: astro-ph
25
The categories of this paper are: patt-sol
15
The categories of this paper are: nucl-ex
7
The categories of this paper are: hep-th
21
The categories of this paper are: adap-org
34
The categories of this paper are: chem-ph
37
The categories of this paper are: supr-con
5
The categories of this paper are: gr-qc
32
The categories of this paper are: atom-ph
20
The categories of this paper are: acc-phys
18
The categories of this paper are: econ
33
The categories of this paper are: plasm-ph
36
The categories of this paper are: cmp-lg
2
The categories of this paper are: math