  Corpus preparation for low-resource languages and for development of human
language technology to analyze or computationally process them is a laborious
task, primarily due to the unavailability of expert linguists who are native
speakers of these languages and also due to the time and resources required.
Bhojpuri, Magahi, and Maithili, languages of the Purvanchal region of India (in
the north-eastern parts), are low-resource languages belonging to the
Indo-Aryan (or Indic) family. They are closely related to Hindi, which is a
relatively high-resource language, which is why we compare with Hindi. We
collected corpora for these three languages from various sources and cleaned
them to the extent possible, without changing the data in them. The text
belongs to different domains and genres. We calculated some basic statistical
measures for these corpora at character, word, syllable, and morpheme levels.
These corpora were also annotated with parts-of-speech (POS) and chunk tags.
The basic statistical measures were both absolute and relative and were
exptected to indicate of linguistic properties such as morphological, lexical,
phonological, and syntactic complexities (or richness). The results were
compared with a standard Hindi corpus. For most of the measures, we tried to
the corpus size the same across the languages to avoid the effect of corpus
size, but in some cases it turned out that using the full corpus was better,
even if sizes were very different. Although the results are not very clear, we
try to draw some conclusions about the languages and the corpora. For POS
tagging and chunking, the BIS tagset was used to manually annotate the data.
The POS tagged data sizes are 16067, 14669 and 12310 sentences, respectively,
for Bhojpuri, Magahi and Maithili. The sizes for chunking are 9695 and 1954
sentences for Bhojpuri and Maithili, respectively.
