Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Advanced Dropout: A Model-free Methodology for Bayesian Dropout
  Optimization   Due to lack of data, overfitting ubiquitously exists in real-world
applications of deep neural networks (DNNs). We propose advanced dropout, a
model-free methodology, to mitigate overfitting and improve the performance of
DNNs. The advanced dropout technique applies a model-free and easily
implemented distribution with parametric prior, and adaptively adjusts dropout
rate. Specifically, the distribution parameters are optimized by stochastic
gradient variational Bayes in order to carry out an end-to-end training. We
evaluate the effectiveness of the advanced dropout against nine dropout
techniques on seven computer vision datasets (five small-scale datasets and two
large-scale datasets) with various base models. The advanced dropout
outperforms all the referred techniques on all the datasets.We further compare
the effectiveness ratios and find that advanced dropout achieves the highest
one on most cases. Next, we conduct a set of analysis of dropout rate
characteristics, including convergence of the adaptive dropout rate, the
learned distributions of dropout masks, and a comparison with dropout rate
generation without an explicit distribution. In addition, the ability of
overfitting prevention is evaluated and confirmed. Finally, we extend the
application of the advanced dropout to uncertainty inference, network pruning,
text classification, and regression. The proposed advanced dropout is also
superior to the corresponding referred methods. Codes are available at
https://github.com/PRIS-CV/AdvancedDropout.

19
The categories of this paper are: eess
11
The categories of this paper are: hep-lat
4
The categories of this paper are: cond-mat
29
The categories of this paper are: funct-an
24
The categories of this paper are: dg-ga
31
The categories of this paper are: ao-sci
30
The categories of this paper are: q-alg
36
The categories of this paper are: cmp-lg
17
The categories of this paper are: q-fin
27
The categories of this paper are: comp-gas
5
The categories of this paper are: gr-qc
9
The categories of this paper are: nlin
20
The categories of this paper are: acc-phys
13
The categories of this paper are: quant-ph
28
The categories of this paper are: alg-geom
35
The categories of this paper are: mtrl-th