Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Coordinating Filters for Faster Deep Neural Networks   Very large-scale Deep Neural Networks (DNNs) have achieved remarkable
successes in a large variety of computer vision tasks. However, the high
computation intensity of DNNs makes it challenging to deploy these models on
resource-limited systems. Some studies used low-rank approaches that
approximate the filters by low-rank basis to accelerate the testing. Those
works directly decomposed the pre-trained DNNs by Low-Rank Approximations
(LRA). How to train DNNs toward lower-rank space for more efficient DNNs,
however, remains as an open area. To solve the issue, in this work, we propose
Force Regularization, which uses attractive forces to enforce filters so as to
coordinate more weight information into lower-rank space. We mathematically and
empirically verify that after applying our technique, standard LRA methods can
reconstruct filters using much lower basis and thus result in faster DNNs. The
effectiveness of our approach is comprehensively evaluated in ResNets, AlexNet,
and GoogLeNet. In AlexNet, for example, Force Regularization gains 2x speedup
on modern GPU without accuracy loss and 4.05x speedup on CPU by paying small
accuracy degradation. Moreover, Force Regularization better initializes the
low-rank DNNs such that the fine-tuning can converge faster toward higher
accuracy. The obtained lower-rank DNNs can be further sparsified, proving that
Force Regularization can be integrated with state-of-the-art sparsity-based
acceleration methods. Source code is available in
https://github.com/wenwei202/caffe

16
The categories of this paper are: stat
24
The categories of this paper are: dg-ga
37
The categories of this paper are: supr-con
29
The categories of this paper are: funct-an
7
The categories of this paper are: hep-th
22
The categories of this paper are: chao-dyn
2
The categories of this paper are: math
6
The categories of this paper are: astro-ph
9
The categories of this paper are: nlin
11
The categories of this paper are: hep-lat
26
The categories of this paper are: bayes-an
10
The categories of this paper are: q-bio
8
The categories of this paper are: hep-ex
20
The categories of this paper are: acc-phys
32
The categories of this paper are: atom-ph
15
The categories of this paper are: nucl-ex