Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Critical Hyper-Parameters: No Random, No Cry   The selection of hyper-parameters is critical in Deep Learning. Because of
the long training time of complex models and the availability of compute
resources in the cloud, "one-shot" optimization schemes - where the sets of
hyper-parameters are selected in advance (e.g. on a grid or in a random manner)
and the training is executed in parallel - are commonly used. It is known that
grid search is sub-optimal, especially when only a few critical parameters
matter, and suggest to use random search instead. Yet, random search can be
"unlucky" and produce sets of values that leave some part of the domain
unexplored. Quasi-random methods, such as Low Discrepancy Sequences (LDS) avoid
these issues. We show that such methods have theoretical properties that make
them appealing for performing hyperparameter search, and demonstrate that, when
applied to the selection of hyperparameters of complex Deep Learning models
(such as state-of-the-art LSTM language models and image classification
models), they yield suitable hyperparameters values with much fewer runs than
random search. We propose a particularly simple LDS method which can be used as
a drop-in replacement for grid or random search in any Deep Learning pipeline,
both as a fully one-shot hyperparameter search or as an initializer in
iterative batch optimization.

2
The categories of this paper are: math
15
The categories of this paper are: nucl-ex
16
The categories of this paper are: stat
12
The categories of this paper are: nucl-th
4
The categories of this paper are: cond-mat
30
The categories of this paper are: q-alg
18
The categories of this paper are: econ
24
The categories of this paper are: dg-ga
19
The categories of this paper are: eess
23
The categories of this paper are: solv-int
21
The categories of this paper are: adap-org
9
The categories of this paper are: nlin
10
The categories of this paper are: q-bio
36
The categories of this paper are: cmp-lg
3
The categories of this paper are: physics
37
The categories of this paper are: supr-con