Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Nuanced Metrics for Measuring Unintended Bias with Real Data for Text
  Classification   Unintended bias in Machine Learning can manifest as systemic differences in
performance for different demographic groups, potentially compounding existing
challenges to fairness in society at large. In this paper, we introduce a suite
of threshold-agnostic metrics that provide a nuanced view of this unintended
bias, by considering the various ways that a classifier's score distribution
can vary across designated groups. We also introduce a large new test set of
online comments with crowd-sourced annotations for identity references. We use
this to show how our metrics can be used to find new and potentially subtle
unintended bias in existing public models.

9
The categories of this paper are: nlin
20
The categories of this paper are: acc-phys
24
The categories of this paper are: dg-ga
14
The categories of this paper are: math-ph
27
The categories of this paper are: comp-gas
19
The categories of this paper are: eess
6
The categories of this paper are: astro-ph
12
The categories of this paper are: nucl-th
35
The categories of this paper are: mtrl-th
32
The categories of this paper are: atom-ph
22
The categories of this paper are: chao-dyn
4
The categories of this paper are: cond-mat
0
The categories of this paper are: hep-ph
34
The categories of this paper are: chem-ph
28
The categories of this paper are: alg-geom
25
The categories of this paper are: patt-sol