Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: MixMix: All You Need for Data-Free Compression Are Feature and Data
  Mixing   User data confidentiality protection is becoming a rising challenge in the
present deep learning research. Without access to data, conventional
data-driven model compression faces a higher risk of performance degradation.
Recently, some works propose to generate images from a specific pretrained
model to serve as training data. However, the inversion process only utilizes
biased feature statistics stored in one model and is from low-dimension to
high-dimension. As a consequence, it inevitably encounters the difficulties of
generalizability and inexact inversion, which leads to unsatisfactory
performance. To address these problems, we propose MixMix based on two simple
yet effective techniques: (1) Feature Mixing: utilizes various models to
construct a universal feature space for generalized inversion; (2) Data Mixing:
mixes the synthesized images and labels to generate exact label information. We
prove the effectiveness of MixMix from both theoretical and empirical
perspectives. Extensive experiments show that MixMix outperforms existing
methods on the mainstream compression tasks, including quantization, knowledge
distillation, and pruning. Specifically, MixMix achieves up to 4% and 20%
accuracy uplift on quantization and pruning, respectively, compared to existing
data-free compression work.

0
The categories of this paper are: hep-ph
37
The categories of this paper are: supr-con
19
The categories of this paper are: eess
36
The categories of this paper are: cmp-lg
3
The categories of this paper are: physics
7
The categories of this paper are: hep-th
24
The categories of this paper are: dg-ga
18
The categories of this paper are: econ
31
The categories of this paper are: ao-sci
8
The categories of this paper are: hep-ex
28
The categories of this paper are: alg-geom
14
The categories of this paper are: math-ph
11
The categories of this paper are: hep-lat
10
The categories of this paper are: q-bio
5
The categories of this paper are: gr-qc
2
The categories of this paper are: math