Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Compressing Heavy-Tailed Weight Matrices for Non-Vacuous Generalization
  Bounds   Heavy-tailed distributions have been studied in statistics, random matrix
theory, physics, and econometrics as models of correlated systems, among other
domains. Further, heavy-tail distributed eigenvalues of the covariance matrix
of the weight matrices in neural networks have been shown to empirically
correlate with test set accuracy in several works (e.g. arXiv:1901.08276), but
a formal relationship between heavy-tail distributed parameters and
generalization bounds was yet to be demonstrated. In this work, the compression
framework of arXiv:1802.05296 is utilized to show that matrices with heavy-tail
distributed matrix elements can be compressed, resulting in networks with
sparse weight matrices. Since the parameter count has been reduced to a sum of
the non-zero elements of sparse matrices, the compression framework allows us
to bound the generalization gap of the resulting compressed network with a
non-vacuous generalization bound. Further, the action of these matrices on a
vector is discussed, and how they may relate to compression and resilient
classification is analyzed.

14
The categories of this paper are: math-ph
7
The categories of this paper are: hep-th
32
The categories of this paper are: atom-ph
2
The categories of this paper are: math
25
The categories of this paper are: patt-sol
0
The categories of this paper are: hep-ph
31
The categories of this paper are: ao-sci
33
The categories of this paper are: plasm-ph
26
The categories of this paper are: bayes-an
30
The categories of this paper are: q-alg
9
The categories of this paper are: nlin
12
The categories of this paper are: nucl-th
20
The categories of this paper are: acc-phys
28
The categories of this paper are: alg-geom
3
The categories of this paper are: physics
11
The categories of this paper are: hep-lat