Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Revisiting Loss Modelling for Unstructured Pruning   By removing parameters from deep neural networks, unstructured pruning
methods aim at cutting down memory footprint and computational cost, while
maintaining prediction accuracy. In order to tackle this otherwise intractable
problem, many of these methods model the loss landscape using first or second
order Taylor expansions to identify which parameters can be discarded. We
revisit loss modelling for unstructured pruning: we show the importance of
ensuring locality of the pruning steps. We systematically compare first and
second order Taylor expansions and empirically show that both can reach similar
levels of performance. Finally, we show that better preserving the original
network function does not necessarily transfer to better performing networks
after fine-tuning, suggesting that only considering the impact of pruning on
the loss might not be a sufficient objective to design good pruning criteria.

14
The categories of this paper are: math-ph
31
The categories of this paper are: ao-sci
5
The categories of this paper are: gr-qc
37
The categories of this paper are: supr-con
34
The categories of this paper are: chem-ph
15
The categories of this paper are: nucl-ex
12
The categories of this paper are: nucl-th
30
The categories of this paper are: q-alg
19
The categories of this paper are: eess
0
The categories of this paper are: hep-ph
18
The categories of this paper are: econ
26
The categories of this paper are: bayes-an
22
The categories of this paper are: chao-dyn
11
The categories of this paper are: hep-lat
13
The categories of this paper are: quant-ph
32
The categories of this paper are: atom-ph