Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Unsupervised Distillation of Syntactic Information from Contextualized
  Word Representations   Contextualized word representations, such as ELMo and BERT, were shown to
perform well on various semantic and syntactic tasks. In this work, we tackle
the task of unsupervised disentanglement between semantics and structure in
neural language representations: we aim to learn a transformation of the
contextualized vectors, that discards the lexical semantics, but keeps the
structural information. To this end, we automatically generate groups of
sentences which are structurally similar but semantically different, and use
metric-learning approach to learn a transformation that emphasizes the
structural component that is encoded in the vectors. We demonstrate that our
transformation clusters vectors in space by structural properties, rather than
by lexical semantics. Finally, we demonstrate the utility of our distilled
representations by showing that they outperform the original contextualized
representations in a few-shot parsing setting.

29
The categories of this paper are: funct-an
7
The categories of this paper are: hep-th
37
The categories of this paper are: supr-con
20
The categories of this paper are: acc-phys
34
The categories of this paper are: chem-ph
2
The categories of this paper are: math
13
The categories of this paper are: quant-ph
27
The categories of this paper are: comp-gas
22
The categories of this paper are: chao-dyn
12
The categories of this paper are: nucl-th
33
The categories of this paper are: plasm-ph
26
The categories of this paper are: bayes-an
16
The categories of this paper are: stat
17
The categories of this paper are: q-fin
4
The categories of this paper are: cond-mat
10
The categories of this paper are: q-bio