Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: SoftTarget Regularization: An Effective Technique to Reduce Over-Fitting
  in Neural Networks   Deep neural networks are learning models with a very high capacity and
therefore prone to over-fitting. Many regularization techniques such as
Dropout, DropConnect, and weight decay all attempt to solve the problem of
over-fitting by reducing the capacity of their respective models (Srivastava et
al., 2014), (Wan et al., 2013), (Krogh & Hertz, 1992). In this paper we
introduce a new form of regularization that guides the learning problem in a
way that reduces over-fitting without sacrificing the capacity of the model.
The mistakes that models make in early stages of training carry information
about the learning problem. By adjusting the labels of the current epoch of
training through a weighted average of the real labels, and an exponential
average of the past soft-targets we achieved a regularization scheme as
powerful as Dropout without necessarily reducing the capacity of the model, and
simplified the complexity of the learning problem. SoftTarget regularization
proved to be an effective tool in various neural network architectures.

26
The categories of this paper are: bayes-an
31
The categories of this paper are: ao-sci
33
The categories of this paper are: plasm-ph
36
The categories of this paper are: cmp-lg
23
The categories of this paper are: solv-int
4
The categories of this paper are: cond-mat
37
The categories of this paper are: supr-con
35
The categories of this paper are: mtrl-th
12
The categories of this paper are: nucl-th
2
The categories of this paper are: math
32
The categories of this paper are: atom-ph
28
The categories of this paper are: alg-geom
17
The categories of this paper are: q-fin
3
The categories of this paper are: physics
30
The categories of this paper are: q-alg
29
The categories of this paper are: funct-an