Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: SA-GD: Improved Gradient Descent Learning Strategy with Simulated
  Annealing   Gradient descent algorithm is the most utilized method when optimizing
machine learning issues. However, there exists many local minimums and saddle
points in the loss function, especially for high dimensional non-convex
optimization problems like deep learning. Gradient descent may make loss
function trapped in these local intervals which impedes further optimization,
resulting in poor generalization ability. This paper proposes the SA-GD
algorithm which introduces the thought of simulated annealing algorithm to
gradient descent. SA-GD method offers model the ability of mounting hills in
probability, tending to enable the model to jump out of these local areas and
converge to a optimal state finally. We took CNN models as an example and
tested the basic CNN models on various benchmark datasets. Compared to the
baseline models with traditional gradient descent algorithm, models with SA-GD
algorithm possess better generalization ability without sacrificing the
efficiency and stability of model convergence. In addition, SA-GD can be
utilized as an effective ensemble learning approach which improves the final
performance significantly.

22
The categories of this paper are: chao-dyn
17
The categories of this paper are: q-fin
32
The categories of this paper are: atom-ph
19
The categories of this paper are: eess
3
The categories of this paper are: physics
25
The categories of this paper are: patt-sol
35
The categories of this paper are: mtrl-th
21
The categories of this paper are: adap-org
16
The categories of this paper are: stat
7
The categories of this paper are: hep-th
28
The categories of this paper are: alg-geom
20
The categories of this paper are: acc-phys
15
The categories of this paper are: nucl-ex
24
The categories of this paper are: dg-ga
34
The categories of this paper are: chem-ph
4
The categories of this paper are: cond-mat