Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: An Investigation on Deep Learning with Beta Stabilizer   Artificial neural networks (ANN) have been used in many applications such
like handwriting recognition and speech recognition. It is well-known that
learning rate is a crucial value in the training procedure for artificial
neural networks. It is shown that the initial value of learning rate can
confoundedly affect the final result and this value is always set manually in
practice. A new parameter called beta stabilizer has been introduced to reduce
the sensitivity of the initial learning rate. But this method has only been
proposed for deep neural network (DNN) with sigmoid activation function. In
this paper we extended beta stabilizer to long short-term memory (LSTM) and
investigated the effects of beta stabilizer parameters on different models,
including LSTM and DNN with relu activation function. It is concluded that beta
stabilizer parameters can reduce the sensitivity of learning rate with almost
the same performance on DNN with relu activation function and LSTM. However, it
is shown that the effects of beta stabilizer on DNN with relu activation
function and LSTM are fewer than the effects on DNN with sigmoid activation
function.

30
The categories of this paper are: q-alg
27
The categories of this paper are: comp-gas
26
The categories of this paper are: bayes-an
20
The categories of this paper are: acc-phys
19
The categories of this paper are: eess
24
The categories of this paper are: dg-ga
14
The categories of this paper are: math-ph
34
The categories of this paper are: chem-ph
37
The categories of this paper are: supr-con
13
The categories of this paper are: quant-ph
6
The categories of this paper are: astro-ph
16
The categories of this paper are: stat
3
The categories of this paper are: physics
2
The categories of this paper are: math
17
The categories of this paper are: q-fin
28
The categories of this paper are: alg-geom