Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Implicit differentiation of Lasso-type models for hyperparameter
  optimization   Setting regularization parameters for Lasso-type estimators is notoriously
difficult, though crucial in practice. The most popular hyperparameter
optimization approach is grid-search using held-out validation data.
Grid-search however requires to choose a predefined grid for each parameter,
which scales exponentially in the number of parameters. Another approach is to
cast hyperparameter optimization as a bi-level optimization problem, one can
solve by gradient descent. The key challenge for these methods is the
estimation of the gradient with respect to the hyperparameters. Computing this
gradient via forward or backward automatic differentiation is possible yet
usually suffers from high memory consumption. Alternatively implicit
differentiation typically involves solving a linear system which can be
prohibitive and numerically unstable in high dimension. In addition, implicit
differentiation usually assumes smooth loss functions, which is not the case
for Lasso-type problems. This work introduces an efficient implicit
differentiation algorithm, without matrix inversion, tailored for Lasso-type
problems. Our approach scales to high-dimensional data by leveraging the
sparsity of the solutions. Experiments demonstrate that the proposed method
outperforms a large number of standard methods to optimize the error on
held-out data, or the Stein Unbiased Risk Estimator (SURE).

9
The categories of this paper are: nlin
22
The categories of this paper are: chao-dyn
3
The categories of this paper are: physics
11
The categories of this paper are: hep-lat
36
The categories of this paper are: cmp-lg
8
The categories of this paper are: hep-ex
25
The categories of this paper are: patt-sol
31
The categories of this paper are: ao-sci
30
The categories of this paper are: q-alg
14
The categories of this paper are: math-ph
28
The categories of this paper are: alg-geom
2
The categories of this paper are: math
24
The categories of this paper are: dg-ga
17
The categories of this paper are: q-fin
4
The categories of this paper are: cond-mat
29
The categories of this paper are: funct-an