Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Dynamic Automatic Differentiation of GPU Broadcast Kernels   We show how forward-mode automatic differentiation (AD) can be employed
within larger reverse-mode computations to dynamically differentiate broadcast
operations in a GPU-friendly manner. Our technique fully exploits the broadcast
Jacobian's inherent sparsity structure, and unlike a pure reverse-mode
approach, this "mixed-mode" approach does not require a backwards pass over the
broadcasted operation's subgraph, obviating the need for several
reverse-mode-specific programmability restrictions on user-authored broadcast
operations. Most notably, this approach allows broadcast fusion in primal code
despite the presence of data-dependent control flow. We discuss an experiment
in which a Julia implementation of our technique outperformed pure reverse-mode
TensorFlow and Julia implementations for differentiating through broadcast
operations within an HM-LSTM cell update calculation.

8
The categories of this paper are: hep-ex
0
The categories of this paper are: hep-ph
10
The categories of this paper are: q-bio
30
The categories of this paper are: q-alg
4
The categories of this paper are: cond-mat
28
The categories of this paper are: alg-geom
19
The categories of this paper are: eess
13
The categories of this paper are: quant-ph
29
The categories of this paper are: funct-an
17
The categories of this paper are: q-fin
33
The categories of this paper are: plasm-ph
36
The categories of this paper are: cmp-lg
20
The categories of this paper are: acc-phys
31
The categories of this paper are: ao-sci
34
The categories of this paper are: chem-ph
6
The categories of this paper are: astro-ph