Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: MUSE: Feature Self-Distillation with Mutual Information and
  Self-Information   We present a novel information-theoretic approach to introduce dependency
among features of a deep convolutional neural network (CNN). The core idea of
our proposed method, called MUSE, is to combine MUtual information and
SElf-information to jointly improve the expressivity of all features extracted
from different layers in a CNN. We present two variants of the realization of
MUSE -- Additive Information and Multiplicative Information. Importantly, we
argue and empirically demonstrate that MUSE, compared to other feature
discrepancy functions, is a more functional proxy to introduce dependency and
effectively improve the expressivity of all features in the knowledge
distillation framework. MUSE achieves superior performance over a variety of
popular architectures and feature discrepancy functions for self-distillation
and online distillation, and performs competitively with the state-of-the-art
methods for offline distillation. MUSE is also demonstrably versatile that
enables it to be easily extended to CNN-based models on tasks other than image
classification such as object detection.

21
The categories of this paper are: adap-org
16
The categories of this paper are: stat
5
The categories of this paper are: gr-qc
36
The categories of this paper are: cmp-lg
10
The categories of this paper are: q-bio
14
The categories of this paper are: math-ph
3
The categories of this paper are: physics
17
The categories of this paper are: q-fin
26
The categories of this paper are: bayes-an
8
The categories of this paper are: hep-ex
20
The categories of this paper are: acc-phys
11
The categories of this paper are: hep-lat
23
The categories of this paper are: solv-int
33
The categories of this paper are: plasm-ph
28
The categories of this paper are: alg-geom
13
The categories of this paper are: quant-ph