Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Unsupervised Learning of Sentence Representations Using Sequence
  Consistency   Computing universal distributed representations of sentences is a fundamental
task in natural language processing. We propose ConsSent, a simple yet
surprisingly powerful unsupervised method to learn such representations by
enforcing consistency constraints on sequences of tokens. We consider two
classes of such constraints -- sequences that form a sentence and between two
sequences that form a sentence when merged. We learn sentence encoders by
training them to distinguish between consistent and inconsistent examples, the
latter being generated by randomly perturbing consistent examples in six
different ways. Extensive evaluation on several transfer learning and
linguistic probing tasks shows improved performance over strong unsupervised
and supervised baselines, substantially surpassing them in several cases. Our
best results are achieved by training sentence encoders in a multitask setting
and by an ensemble of encoders trained on the individual tasks.

11
The categories of this paper are: hep-lat
4
The categories of this paper are: cond-mat
15
The categories of this paper are: nucl-ex
32
The categories of this paper are: atom-ph
21
The categories of this paper are: adap-org
5
The categories of this paper are: gr-qc
27
The categories of this paper are: comp-gas
31
The categories of this paper are: ao-sci
25
The categories of this paper are: patt-sol
9
The categories of this paper are: nlin
26
The categories of this paper are: bayes-an
24
The categories of this paper are: dg-ga
33
The categories of this paper are: plasm-ph
22
The categories of this paper are: chao-dyn
20
The categories of this paper are: acc-phys
17
The categories of this paper are: q-fin