Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: The Woman Worked as a Babysitter: On Biases in Language Generation   We present a systematic study of biases in natural language generation (NLG)
by analyzing text generated from prompts that contain mentions of different
demographic groups. In this work, we introduce the notion of the regard towards
a demographic, use the varying levels of regard towards different demographics
as a defining metric for bias in NLG, and analyze the extent to which sentiment
scores are a relevant proxy metric for regard. To this end, we collect
strategically-generated text from language models and manually annotate the
text with both sentiment and regard scores. Additionally, we build an automatic
regard classifier through transfer learning, so that we can analyze biases in
unseen text. Together, these methods reveal the extent of the biased nature of
language model generations. Our analysis provides a study of biases in NLG,
bias metrics and correlated human judgments, and empirical evidence on the
usefulness of our annotated dataset.

27
The categories of this paper are: comp-gas
34
The categories of this paper are: chem-ph
22
The categories of this paper are: chao-dyn
35
The categories of this paper are: mtrl-th
33
The categories of this paper are: plasm-ph
4
The categories of this paper are: cond-mat
7
The categories of this paper are: hep-th
31
The categories of this paper are: ao-sci
3
The categories of this paper are: physics
19
The categories of this paper are: eess
26
The categories of this paper are: bayes-an
24
The categories of this paper are: dg-ga
36
The categories of this paper are: cmp-lg
23
The categories of this paper are: solv-int
20
The categories of this paper are: acc-phys
14
The categories of this paper are: math-ph