Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Towards a Theoretical Understanding of Hashing-Based Neural Nets   Parameter reduction has been an important topic in deep learning due to the
ever-increasing size of deep neural network models and the need to train and
run them on resource limited machines. Despite many efforts in this area, there
were no rigorous theoretical guarantees on why existing neural net compression
methods should work. In this paper, we provide provable guarantees on some
hashing-based parameter reduction methods in neural nets. First, we introduce a
neural net compression scheme based on random linear sketching (which is
usually implemented efficiently via hashing), and show that the sketched
(smaller) network is able to approximate the original network on all input data
coming from any smooth and well-conditioned low-dimensional manifold. The
sketched network can also be trained directly via back-propagation. Next, we
study the previously proposed HashedNets architecture and show that the
optimization landscape of one-hidden-layer HashedNets has a local strong
convexity property similar to a normal fully connected neural network. We
complement our theoretical results with empirical verifications.

0
The categories of this paper are: hep-ph
36
The categories of this paper are: cmp-lg
23
The categories of this paper are: solv-int
19
The categories of this paper are: eess
32
The categories of this paper are: atom-ph
11
The categories of this paper are: hep-lat
15
The categories of this paper are: nucl-ex
27
The categories of this paper are: comp-gas
10
The categories of this paper are: q-bio
25
The categories of this paper are: patt-sol
28
The categories of this paper are: alg-geom
18
The categories of this paper are: econ
24
The categories of this paper are: dg-ga
13
The categories of this paper are: quant-ph
34
The categories of this paper are: chem-ph
2
The categories of this paper are: math