Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: TURINGBENCH: A Benchmark Environment for Turing Test in the Age of
  Neural Text Generation   Recent progress in generative language models has enabled machines to
generate astonishingly realistic texts. While there are many legitimate
applications of such models, there is also a rising need to distinguish
machine-generated texts from human-written ones (e.g., fake news detection).
However, to our best knowledge, there is currently no benchmark environment
with datasets and tasks to systematically study the so-called "Turing Test"
problem for neural text generation methods. In this work, we present the
TuringBench benchmark environment, which is comprised of (1) a dataset with
200K human- or machine-generated samples across 20 labels {Human, GPT-1,
GPT-2_small, GPT-2_medium, GPT-2_large, GPT-2_xl, GPT-2_PyTorch, GPT-3,
GROVER_base, GROVER_large, GROVER_mega, CTRL, XLM, XLNET_base, XLNET_large,
FAIR_wmt19, FAIR_wmt20, TRANSFORMER_XL, PPLM_distil, PPLM_gpt2}, (2) two
benchmark tasks -- i.e., Turing Test (TT) and Authorship Attribution (AA), and
(3) a website with leaderboards. Our preliminary experimental results using
TuringBench show that FAIR_wmt20 and GPT-3 are the current winners, among all
language models tested, in generating the most human-like indistinguishable
texts with the lowest F1 score by five state-of-the-art TT detection models.
The TuringBench is available at: https://turingbench.ist.psu.edu/

23
The categories of this paper are: solv-int
0
The categories of this paper are: hep-ph
6
The categories of this paper are: astro-ph
24
The categories of this paper are: dg-ga
17
The categories of this paper are: q-fin
32
The categories of this paper are: atom-ph
33
The categories of this paper are: plasm-ph
30
The categories of this paper are: q-alg
9
The categories of this paper are: nlin
12
The categories of this paper are: nucl-th
18
The categories of this paper are: econ
5
The categories of this paper are: gr-qc
36
The categories of this paper are: cmp-lg
37
The categories of this paper are: supr-con
11
The categories of this paper are: hep-lat
7
The categories of this paper are: hep-th