Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Mono vs Multilingual BERT for Hate Speech Detection and Text
  Classification: A Case Study in Marathi   Transformers are the most eminent architectures used for a vast range of
Natural Language Processing tasks. These models are pre-trained over a large
text corpus and are meant to serve state-of-the-art results over tasks like
text classification. In this work, we conduct a comparative study between
monolingual and multilingual BERT models. We focus on the Marathi language and
evaluate the models on the datasets for hate speech detection, sentiment
analysis and simple text classification in Marathi. We use standard
multilingual models such as mBERT, indicBERT and xlm-RoBERTa and compare with
MahaBERT, MahaALBERT and MahaRoBERTa, the monolingual models for Marathi. We
further show that Marathi monolingual models outperform the multilingual BERT
variants on five different downstream fine-tuning experiments. We also evaluate
sentence embeddings from these models by freezing the BERT encoder layers. We
show that monolingual MahaBERT based models provide rich representations as
compared to sentence embeddings from multi-lingual counterparts. However, we
observe that these embeddings are not generic enough and do not work well on
out of domain social media datasets. We consider two Marathi hate speech
datasets L3Cube-MahaHate, HASOC-2021, a Marathi sentiment classification
dataset L3Cube-MahaSent, and Marathi Headline, Articles classification
datasets.

4
The categories of this paper are: cond-mat
15
The categories of this paper are: nucl-ex
2
The categories of this paper are: math
37
The categories of this paper are: supr-con
29
The categories of this paper are: funct-an
26
The categories of this paper are: bayes-an
13
The categories of this paper are: quant-ph
28
The categories of this paper are: alg-geom
23
The categories of this paper are: solv-int
36
The categories of this paper are: cmp-lg
18
The categories of this paper are: econ
21
The categories of this paper are: adap-org
22
The categories of this paper are: chao-dyn
6
The categories of this paper are: astro-ph
17
The categories of this paper are: q-fin
3
The categories of this paper are: physics