Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Refining BERT Embeddings for Document Hashing via Mutual Information
  Maximization   Existing unsupervised document hashing methods are mostly established on
generative models. Due to the difficulties of capturing long dependency
structures, these methods rarely model the raw documents directly, but instead
to model the features extracted from them (e.g. bag-of-words (BOW), TFIDF). In
this paper, we propose to learn hash codes from BERT embeddings after observing
their tremendous successes on downstream tasks. As a first try, we modify
existing generative hashing models to accommodate the BERT embeddings. However,
little improvement is observed over the codes learned from the old BOW or TFIDF
features. We attribute this to the reconstruction requirement in the generative
hashing, which will enforce irrelevant information that is abundant in the BERT
embeddings also compressed into the codes. To remedy this issue, a new
unsupervised hashing paradigm is further proposed based on the mutual
information (MI) maximization principle. Specifically, the method first
constructs appropriate global and local codes from the documents and then seeks
to maximize their mutual information. Experimental results on three benchmark
datasets demonstrate that the proposed method is able to generate hash codes
that outperform existing ones learned from BOW features by a substantial
margin.

21
The categories of this paper are: adap-org
11
The categories of this paper are: hep-lat
3
The categories of this paper are: physics
16
The categories of this paper are: stat
12
The categories of this paper are: nucl-th
15
The categories of this paper are: nucl-ex
8
The categories of this paper are: hep-ex
7
The categories of this paper are: hep-th
23
The categories of this paper are: solv-int
0
The categories of this paper are: hep-ph
20
The categories of this paper are: acc-phys
32
The categories of this paper are: atom-ph
29
The categories of this paper are: funct-an
31
The categories of this paper are: ao-sci
2
The categories of this paper are: math
4
The categories of this paper are: cond-mat