Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: A Methodology for Automatic Selection of Activation Functions to Design
  Hybrid Deep Neural Networks   Activation functions influence behavior and performance of DNNs. Nonlinear
activation functions, like Rectified Linear Units (ReLU), Exponential Linear
Units (ELU) and Scaled Exponential Linear Units (SELU), outperform the linear
counterparts. However, selecting an appropriate activation function is a
challenging problem, as it affects the accuracy and the complexity of the given
DNN. In this paper, we propose a novel methodology to automatically select the
best-possible activation function for each layer of a given DNN, such that the
overall DNN accuracy, compared to considering only one type of activation
function for the whole DNN, is improved. However, an associated scientific
challenge in exploring all the different configurations of activation functions
would be time and resource-consuming. Towards this, our methodology identifies
the Evaluation Points during learning to evaluate the accuracy in an
intermediate step of training and to perform early termination by checking the
accuracy gradient of the learning curve. This helps in significantly reducing
the exploration time during training. Moreover, our methodology selects, for
each layer, the dropout rate that optimizes the accuracy. Experiments show that
we are able to achieve on average 7% to 15% Relative Error Reduction on MNIST,
CIFAR-10 and CIFAR-100 benchmarks, with limited performance and power penalty
on GPUs.

17
The categories of this paper are: q-fin
3
The categories of this paper are: physics
28
The categories of this paper are: alg-geom
7
The categories of this paper are: hep-th
19
The categories of this paper are: eess
14
The categories of this paper are: math-ph
0
The categories of this paper are: hep-ph
30
The categories of this paper are: q-alg
9
The categories of this paper are: nlin
4
The categories of this paper are: cond-mat
10
The categories of this paper are: q-bio
6
The categories of this paper are: astro-ph
25
The categories of this paper are: patt-sol
11
The categories of this paper are: hep-lat
35
The categories of this paper are: mtrl-th
27
The categories of this paper are: comp-gas