Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Brief Announcement: On the Limits of Parallelizing Convolutional Neural
  Networks on GPUs   GPUs are currently the platform of choice for training neural networks.
However, training a deep neural network (DNN) is a time-consuming process even
on GPUs because of the massive number of parameters that have to be learned. As
a result, accelerating DNN training has been an area of significant research in
the last couple of years.
  While earlier networks such as AlexNet had a linear dependency between layers
and operations, state-of-the-art networks such as ResNet, PathNet, and
GoogleNet have a non-linear structure that exhibits a higher level of
inter-operation parallelism. However, popular deep learning (DL) frameworks
such as TensorFlow and PyTorch launch the majority of neural network
operations, especially convolutions, serially on GPUs and do not exploit this
inter-op parallelism. In this brief announcement, we make a case for the need
and potential benefit of exploiting this rich parallelism in state-of-the-art
non-linear networks for reducing the training time. We identify the challenges
and limitations in enabling concurrent layer execution on GPU backends (such as
cuDNN) of DL frameworks and propose potential solutions.

19
The categories of this paper are: eess
37
The categories of this paper are: supr-con
10
The categories of this paper are: q-bio
36
The categories of this paper are: cmp-lg
14
The categories of this paper are: math-ph
0
The categories of this paper are: hep-ph
16
The categories of this paper are: stat
15
The categories of this paper are: nucl-ex
31
The categories of this paper are: ao-sci
13
The categories of this paper are: quant-ph
17
The categories of this paper are: q-fin
33
The categories of this paper are: plasm-ph
8
The categories of this paper are: hep-ex
4
The categories of this paper are: cond-mat
23
The categories of this paper are: solv-int
25
The categories of this paper are: patt-sol