Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Z-Forcing: Training Stochastic Recurrent Networks   Many efforts have been devoted to training generative latent variable models
with autoregressive decoders, such as recurrent neural networks (RNN).
Stochastic recurrent models have been successful in capturing the variability
observed in natural sequential data such as speech. We unify successful ideas
from recently proposed architectures into a stochastic recurrent model: each
step in the sequence is associated with a latent variable that is used to
condition the recurrent dynamics for future steps. Training is performed with
amortized variational inference where the approximate posterior is augmented
with a RNN that runs backward through the sequence. In addition to maximizing
the variational lower bound, we ease training of the latent variables by adding
an auxiliary cost which forces them to reconstruct the state of the backward
recurrent network. This provides the latent variables with a task-independent
objective that enhances the performance of the overall model. We found this
strategy to perform better than alternative approaches such as KL annealing.
Although being conceptually simple, our model achieves state-of-the-art results
on standard speech benchmarks such as TIMIT and Blizzard and competitive
performance on sequential MNIST. Finally, we apply our model to language
modeling on the IMDB dataset where the auxiliary cost helps in learning
interpretable latent variables. Source Code:
\url{https://github.com/anirudh9119/zforcing_nips17}

30
The categories of this paper are: q-alg
13
The categories of this paper are: quant-ph
37
The categories of this paper are: supr-con
17
The categories of this paper are: q-fin
24
The categories of this paper are: dg-ga
31
The categories of this paper are: ao-sci
20
The categories of this paper are: acc-phys
23
The categories of this paper are: solv-int
2
The categories of this paper are: math
26
The categories of this paper are: bayes-an
33
The categories of this paper are: plasm-ph
19
The categories of this paper are: eess
36
The categories of this paper are: cmp-lg
6
The categories of this paper are: astro-ph
12
The categories of this paper are: nucl-th
5
The categories of this paper are: gr-qc