Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Encoder-Agnostic Adaptation for Conditional Language Generation   Large pretrained language models have changed the way researchers approach
discriminative natural language understanding tasks, leading to the dominance
of approaches that adapt a pretrained model for arbitrary downstream tasks.
However it is an open-question how to use similar techniques for language
generation. Early results in the encoder-agnostic setting have been mostly
negative. In this work we explore methods for adapting a pretrained language
model to arbitrary conditional input. We observe that pretrained transformer
models are sensitive to large parameter changes during tuning. We therefore
propose an adaptation that directly injects arbitrary conditioning into self
attention, an approach we call pseudo self attention. Through experiments on
four diverse conditional text generation tasks we show that this
encoder-agnostic technique outperforms strong baselines, produces coherent
generations, and is data efficient.

13
The categories of this paper are: quant-ph
18
The categories of this paper are: econ
35
The categories of this paper are: mtrl-th
25
The categories of this paper are: patt-sol
34
The categories of this paper are: chem-ph
16
The categories of this paper are: stat
8
The categories of this paper are: hep-ex
20
The categories of this paper are: acc-phys
31
The categories of this paper are: ao-sci
5
The categories of this paper are: gr-qc
3
The categories of this paper are: physics
2
The categories of this paper are: math
36
The categories of this paper are: cmp-lg
15
The categories of this paper are: nucl-ex
29
The categories of this paper are: funct-an
11
The categories of this paper are: hep-lat