Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: How Low Can We Go: Trading Memory for Error in Low-Precision Training   Low-precision arithmetic trains deep learning models using less energy, less
memory and less time. However, we pay a price for the savings: lower precision
may yield larger round-off error and hence larger prediction error. As
applications proliferate, users must choose which precision to use to train a
new model, and chip manufacturers must decide which precisions to manufacture.
We view these precision choices as a hyperparameter tuning problem, and borrow
ideas from meta-learning to learn the tradeoff between memory and error. In
this paper, we introduce Pareto Estimation to Pick the Perfect Precision
(PEPPP). We use matrix factorization to find non-dominated configurations (the
Pareto frontier) with a limited number of network evaluations. For any given
memory budget, the precision that minimizes error is a point on this frontier.
Practitioners can use the frontier to trade memory for error and choose the
best precision for their goals.

25
The categories of this paper are: patt-sol
13
The categories of this paper are: quant-ph
10
The categories of this paper are: q-bio
18
The categories of this paper are: econ
14
The categories of this paper are: math-ph
37
The categories of this paper are: supr-con
4
The categories of this paper are: cond-mat
27
The categories of this paper are: comp-gas
7
The categories of this paper are: hep-th
31
The categories of this paper are: ao-sci
16
The categories of this paper are: stat
35
The categories of this paper are: mtrl-th
22
The categories of this paper are: chao-dyn
21
The categories of this paper are: adap-org
24
The categories of this paper are: dg-ga
0
The categories of this paper are: hep-ph