Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Compression of Fully-Connected Layer in Neural Network by Kronecker
  Product   In this paper we propose and study a technique to reduce the number of
parameters and computation time in fully-connected layers of neural networks
using Kronecker product, at a mild cost of the prediction quality. The
technique proceeds by replacing Fully-Connected layers with so-called Kronecker
Fully-Connected layers, where the weight matrices of the FC layers are
approximated by linear combinations of multiple Kronecker products of smaller
matrices. In particular, given a model trained on SVHN dataset, we are able to
construct a new KFC model with 73\% reduction in total number of parameters,
while the error only rises mildly. In contrast, using low-rank method can only
achieve 35\% reduction in total number of parameters given similar quality
degradation allowance. If we only compare the KFC layer with its counterpart
fully-connected layer, the reduction in the number of parameters exceeds 99\%.
The amount of computation is also reduced as we replace matrix product of the
large matrices in FC layers with matrix products of a few smaller matrices in
KFC layers. Further experiments on MNIST, SVHN and some Chinese Character
recognition models also demonstrate effectiveness of our technique.

13
The categories of this paper are: quant-ph
7
The categories of this paper are: hep-th
29
The categories of this paper are: funct-an
17
The categories of this paper are: q-fin
11
The categories of this paper are: hep-lat
10
The categories of this paper are: q-bio
3
The categories of this paper are: physics
30
The categories of this paper are: q-alg
6
The categories of this paper are: astro-ph
36
The categories of this paper are: cmp-lg
15
The categories of this paper are: nucl-ex
12
The categories of this paper are: nucl-th
31
The categories of this paper are: ao-sci
19
The categories of this paper are: eess
4
The categories of this paper are: cond-mat
14
The categories of this paper are: math-ph