Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Simple Text Detoxification by Identifying a Linear Toxic Subspace in
  Language Model Embeddings   Large pre-trained language models are often trained on large volumes of
internet data, some of which may contain toxic or abusive language.
Consequently, language models encode toxic information, which makes the
real-world usage of these language models limited. Current methods aim to
prevent toxic features from appearing generated text. We hypothesize the
existence of a low-dimensional toxic subspace in the latent space of
pre-trained language models, the existence of which suggests that toxic
features follow some underlying pattern and are thus removable. To construct
this toxic subspace, we propose a method to generalize toxic directions in the
latent space. We also provide a methodology for constructing parallel datasets
using a context based word masking system. Through our experiments, we show
that when the toxic subspace is removed from a set of sentence representations,
almost no toxic representations remain in the result. We demonstrate
empirically that the subspace found using our method generalizes to multiple
toxicity corpora, indicating the existence of a low-dimensional toxic subspace.

31
The categories of this paper are: ao-sci
21
The categories of this paper are: adap-org
0
The categories of this paper are: hep-ph
22
The categories of this paper are: chao-dyn
30
The categories of this paper are: q-alg
24
The categories of this paper are: dg-ga
29
The categories of this paper are: funct-an
11
The categories of this paper are: hep-lat
36
The categories of this paper are: cmp-lg
5
The categories of this paper are: gr-qc
27
The categories of this paper are: comp-gas
17
The categories of this paper are: q-fin
13
The categories of this paper are: quant-ph
12
The categories of this paper are: nucl-th
23
The categories of this paper are: solv-int
4
The categories of this paper are: cond-mat