Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Population-Contrastive-Divergence: Does Consistency help with RBM
  training?   Estimating the log-likelihood gradient with respect to the parameters of a
Restricted Boltzmann Machine (RBM) typically requires sampling using Markov
Chain Monte Carlo (MCMC) techniques. To save computation time, the Markov
chains are only run for a small number of steps, which leads to a biased
estimate. This bias can cause RBM training algorithms such as Contrastive
Divergence (CD) learning to deteriorate. We adopt the idea behind Population
Monte Carlo (PMC) methods to devise a new RBM training algorithm termed
Population-Contrastive-Divergence (pop-CD). Compared to CD, it leads to a
consistent estimate and may have a significantly lower bias. Its computational
overhead is negligible compared to CD. However, the variance of the gradient
estimate increases. We experimentally show that pop-CD can significantly
outperform CD. In many cases, we observed a smaller bias and achieved higher
log-likelihood values. However, when the RBM distribution has many hidden
neurons, the consistent estimate of pop-CD may still have a considerable bias
and the variance of the gradient estimate requires a smaller learning rate.
Thus, despite its superior theoretical properties, it is not advisable to use
pop-CD in its current form on large problems.

20
The categories of this paper are: acc-phys
13
The categories of this paper are: quant-ph
34
The categories of this paper are: chem-ph
31
The categories of this paper are: ao-sci
27
The categories of this paper are: comp-gas
33
The categories of this paper are: plasm-ph
37
The categories of this paper are: supr-con
35
The categories of this paper are: mtrl-th
0
The categories of this paper are: hep-ph
25
The categories of this paper are: patt-sol
6
The categories of this paper are: astro-ph
10
The categories of this paper are: q-bio
36
The categories of this paper are: cmp-lg
15
The categories of this paper are: nucl-ex
8
The categories of this paper are: hep-ex
32
The categories of this paper are: atom-ph