Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Derivatives and residual distribution of regularized M-estimators with
  application to adaptive tuning   This paper studies M-estimators with gradient-Lipschitz loss function
regularized with convex penalty in linear models with Gaussian design matrix
and arbitrary noise distribution. A practical example is the robust M-estimator
constructed with the Huber loss and the Elastic-Net penalty and the noise
distribution has heavy-tails. Our main contributions are three-fold. (i) We
provide general formulae for the derivatives of regularized M-estimators
$\hat\beta(y,X)$ where differentiation is taken with respect to both $y$ and
$X$; this reveals a simple differentiability structure shared by all convex
regularized M-estimators. (ii) Using these derivatives, we characterize the
distribution of the residual $r_i = y_i-x_i^\top\hat\beta$ in the intermediate
high-dimensional regime where dimension and sample size are of the same order.
(iii) Motivated by the distribution of the residuals, we propose a novel
adaptive criterion to select tuning parameters of regularized M-estimators. The
criterion approximates the out-of-sample error up to an additive constant
independent of the estimator, so that minimizing the criterion provides a proxy
for minimizing the out-of-sample error. The proposed adaptive criterion does
not require the knowledge of the noise distribution or of the covariance of the
design. Simulated data confirms the theoretical findings, regarding both the
distribution of the residuals and the success of the criterion as a proxy of
the out-of-sample error. Finally our results reveal new relationships between
the derivatives of $\hat\beta(y,X)$ and the effective degrees of freedom of the
M-estimator, which are of independent interest.

5
The categories of this paper are: gr-qc
29
The categories of this paper are: funct-an
25
The categories of this paper are: patt-sol
37
The categories of this paper are: supr-con
31
The categories of this paper are: ao-sci
20
The categories of this paper are: acc-phys
18
The categories of this paper are: econ
36
The categories of this paper are: cmp-lg
19
The categories of this paper are: eess
9
The categories of this paper are: nlin
4
The categories of this paper are: cond-mat
34
The categories of this paper are: chem-ph
15
The categories of this paper are: nucl-ex
32
The categories of this paper are: atom-ph
24
The categories of this paper are: dg-ga
35
The categories of this paper are: mtrl-th