Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: How to Train Your Neural Network: A Comparative Evaluation   The field of deep learning has witnessed a remarkable shift towards extremely
compute- and memory-intensive neural networks. These newer larger models have
enabled researchers to advance state-of-the-art tools across a variety of
fields. This phenomenon has spurred the development of algorithms for
distributed training of neural networks over a larger number of hardware
accelerators. In this paper, we discuss and compare current state-of-the-art
frameworks for large scale distributed deep learning. First, we survey current
practices in distributed learning and identify the different types of
parallelism used. Then, we present empirical results comparing their
performance on large image and language training tasks. Additionally, we
address their statistical efficiency and memory consumption behavior. Based on
our results, we discuss algorithmic and implementation portions of each
framework which hinder performance.

37
The categories of this paper are: supr-con
10
The categories of this paper are: q-bio
0
The categories of this paper are: hep-ph
2
The categories of this paper are: math
24
The categories of this paper are: dg-ga
17
The categories of this paper are: q-fin
11
The categories of this paper are: hep-lat
29
The categories of this paper are: funct-an
6
The categories of this paper are: astro-ph
3
The categories of this paper are: physics
26
The categories of this paper are: bayes-an
23
The categories of this paper are: solv-int
19
The categories of this paper are: eess
34
The categories of this paper are: chem-ph
7
The categories of this paper are: hep-th
9
The categories of this paper are: nlin