Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Real or Fake? Learning to Discriminate Machine from Human Generated Text   Energy-based models (EBMs), a.k.a. un-normalized models, have had recent
successes in continuous spaces. However, they have not been successfully
applied to model text sequences. While decreasing the energy at training
samples is straightforward, mining (negative) samples where the energy should
be increased is difficult. In part, this is because standard gradient-based
methods are not readily applicable when the input is high-dimensional and
discrete. Here, we side-step this issue by generating negatives using
pre-trained auto-regressive language models. The EBM then works in the residual
of the language model; and is trained to discriminate real text from text
generated by the auto-regressive models. We investigate the generalization
ability of residual EBMs, a pre-requisite for using them in other applications.
We extensively analyze generalization for the task of classifying whether an
input is machine or human generated, a natural task given the training loss and
how we mine negatives. Overall, we observe that EBMs can generalize remarkably
well to changes in the architecture of the generators producing negatives.
However, EBMs exhibit more sensitivity to the training set used by such
generators.

18
The categories of this paper are: econ
31
The categories of this paper are: ao-sci
25
The categories of this paper are: patt-sol
27
The categories of this paper are: comp-gas
5
The categories of this paper are: gr-qc
23
The categories of this paper are: solv-int
15
The categories of this paper are: nucl-ex
3
The categories of this paper are: physics
28
The categories of this paper are: alg-geom
9
The categories of this paper are: nlin
4
The categories of this paper are: cond-mat
17
The categories of this paper are: q-fin
13
The categories of this paper are: quant-ph
8
The categories of this paper are: hep-ex
26
The categories of this paper are: bayes-an
10
The categories of this paper are: q-bio