Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: textTOvec: Deep Contextualized Neural Autoregressive Topic Models of
  Language with Distributed Compositional Prior   We address two challenges of probabilistic topic modelling in order to better
estimate the probability of a word in a given context, i.e., P(word|context):
(1) No Language Structure in Context: Probabilistic topic models ignore word
order by summarizing a given context as a "bag-of-word" and consequently the
semantics of words in the context is lost. The LSTM-LM learns a vector-space
representation of each word by accounting for word order in local collocation
patterns and models complex characteristics of language (e.g., syntax and
semantics), while the TM simultaneously learns a latent representation from the
entire document and discovers the underlying thematic structure. We unite two
complementary paradigms of learning the meaning of word occurrences by
combining a TM (e.g., DocNADE) and a LM in a unified probabilistic framework,
named as ctx-DocNADE. (2) Limited Context and/or Smaller training corpus of
documents: In settings with a small number of word occurrences (i.e., lack of
context) in short text or data sparsity in a corpus of few documents, the
application of TMs is challenging. We address this challenge by incorporating
external knowledge into neural autoregressive topic models via a language
modelling approach: we use word embeddings as input of a LSTM-LM with the aim
to improve the word-topic mapping on a smaller and/or short-text corpus. The
proposed DocNADE extension is named as ctx-DocNADEe.
  We present novel neural autoregressive topic model variants coupled with
neural LMs and embeddings priors that consistently outperform state-of-the-art
generative TMs in terms of generalization (perplexity), interpretability (topic
coherence) and applicability (retrieval and classification) over 6 long-text
and 8 short-text datasets from diverse domains.

21
The categories of this paper are: adap-org
28
The categories of this paper are: alg-geom
24
The categories of this paper are: dg-ga
33
The categories of this paper are: plasm-ph
10
The categories of this paper are: q-bio
6
The categories of this paper are: astro-ph
12
The categories of this paper are: nucl-th
27
The categories of this paper are: comp-gas
8
The categories of this paper are: hep-ex
31
The categories of this paper are: ao-sci
9
The categories of this paper are: nlin
22
The categories of this paper are: chao-dyn
36
The categories of this paper are: cmp-lg
16
The categories of this paper are: stat
15
The categories of this paper are: nucl-ex
19
The categories of this paper are: eess