Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Avoiding The Double Descent Phenomenon of Random Feature Models Using
  Hybrid Regularization   We demonstrate the ability of hybrid regularization methods to automatically
avoid the double descent phenomenon arising in the training of random feature
models (RFM). The hallmark feature of the double descent phenomenon is a spike
in the regularization gap at the interpolation threshold, i.e. when the number
of features in the RFM equals the number of training samples. To close this
gap, the hybrid method considered in our paper combines the respective
strengths of the two most common forms of regularization: early stopping and
weight decay. The scheme does not require hyperparameter tuning as it
automatically selects the stopping iteration and weight decay hyperparameter by
using generalized cross-validation (GCV). This also avoids the necessity of a
dedicated validation set. While the benefits of hybrid methods have been
well-documented for ill-posed inverse problems, our work presents the first use
case in machine learning. To expose the need for regularization and motivate
hybrid methods, we perform detailed numerical experiments inspired by image
classification. In those examples, the hybrid scheme successfully avoids the
double descent phenomenon and yields RFMs whose generalization is comparable
with classical regularization approaches whose hyperparameters are tuned
optimally using the test data. We provide our MATLAB codes for implementing the
numerical experiments in this paper at https://github.com/EmoryMLIP/HybridRFM.

30
The categories of this paper are: q-alg
13
The categories of this paper are: quant-ph
12
The categories of this paper are: nucl-th
15
The categories of this paper are: nucl-ex
11
The categories of this paper are: hep-lat
5
The categories of this paper are: gr-qc
4
The categories of this paper are: cond-mat
35
The categories of this paper are: mtrl-th
28
The categories of this paper are: alg-geom
23
The categories of this paper are: solv-int
26
The categories of this paper are: bayes-an
29
The categories of this paper are: funct-an
22
The categories of this paper are: chao-dyn
36
The categories of this paper are: cmp-lg
24
The categories of this paper are: dg-ga
3
The categories of this paper are: physics