Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Regularized Dynamic Boltzmann Machine with Delay Pruning for
  Unsupervised Learning of Temporal Sequences   We introduce Delay Pruning, a simple yet powerful technique to regularize
dynamic Boltzmann machines (DyBM). The recently introduced DyBM provides a
particularly structured Boltzmann machine, as a generative model of a
multi-dimensional time-series. This Boltzmann machine can have infinitely many
layers of units but allows exact inference and learning based on its
biologically motivated structure. DyBM uses the idea of conduction delays in
the form of fixed length first-in first-out (FIFO) queues, with a neuron
connected to another via this FIFO queue, and spikes from a pre-synaptic neuron
travel along the queue to the post-synaptic neuron with a constant period of
delay. Here, we present Delay Pruning as a mechanism to prune the lengths of
the FIFO queues (making them zero) by setting some delay lengths to one with a
fixed probability, and finally selecting the best performing model with fixed
delays. The uniqueness of structure and a non-sampling based learning rule in
DyBM, make the application of previously proposed regularization techniques
like Dropout or DropConnect difficult, leading to poor generalization. First,
we evaluate the performance of Delay Pruning to let DyBM learn a
multidimensional temporal sequence generated by a Markov chain. Finally, we
show the effectiveness of delay pruning in learning high dimensional sequences
using the moving MNIST dataset, and compare it with Dropout and DropConnect
methods.

34
The categories of this paper are: chem-ph
7
The categories of this paper are: hep-th
28
The categories of this paper are: alg-geom
15
The categories of this paper are: nucl-ex
12
The categories of this paper are: nucl-th
9
The categories of this paper are: nlin
8
The categories of this paper are: hep-ex
23
The categories of this paper are: solv-int
37
The categories of this paper are: supr-con
29
The categories of this paper are: funct-an
4
The categories of this paper are: cond-mat
30
The categories of this paper are: q-alg
11
The categories of this paper are: hep-lat
17
The categories of this paper are: q-fin
18
The categories of this paper are: econ
3
The categories of this paper are: physics