multilinguality:multilingual language:tir language:rmn language:arb language:mxv language:mal language:fij language:som language:cot language:fur language:vie language:zlm language:bam language:chr language:maz language:yad language:ztu language:ykg language:ccp language:alt language:ayr language:njo language:bci language:gyr language:run language:haw language:rgn language:cak language:kwi language:fra language:agr language:duu language:ilo language:nhn language:kdh language:cnh language:bod language:mya language:ady language:pol language:ydd language:cos language:lot language:arl language:glv language:gag language:bfa language:afr language:lij language:ibb language:toi language:tzm language:ron language:ojb language:san language:eng language:bum language:pam language:kqs language:dje language:auc language:smo language:por language:fry language:lad language:pov language:tyv language:guc language:huu language:ese language:kbp language:eve language:yrk language:lin language:tdt language:qvc language:top language:nav language:twi language:oss language:lia language:ame language:hun language:lit language:que language:qug language:nku language:csa language:lao language:knc language:kjh language:jav language:mam language:ita language:ppl language:aar language:tbz language:ssw language:bug language:srp language:kaz language:min language:mad language:orh language:tgk language:kat language:uig language:tzo language:hat language:shn language:kbd language:niv language:idu language:krl language:abk language:mto language:gla language:ijs language:cri language:uzn language:tah language:tob language:kir language:quy language:hnj language:srr language:lvs language:nan language:hns language:snk language:swh language:ekk language:guu language:div language:dzo language:spa language:hms language:ell language:ace language:war language:ind language:cjy language:cfm language:nds language:ewe language:tel language:src language:fuf language:vmw language:zro language:men language:kqn language:nzi language:taj language:khk language:ddn language:nso language:mxi language:pon language:fvr language:hau language:ktu language:tem language:yor language:pnb language:ltz language:evn language:cjs language:nba language:niu language:dan language:acu language:zgh language:chj language:heb language:lua language:quz language:cbi language:cpu language:wuu language:mah language:kmb language:mcd language:ben language:lus language:ajg language:azj language:tha language:dga language:isl language:sus language:fkv language:jiv language:mor language:nio language:als language:buc language:kde language:nbl language:ceb language:ven language:sun language:cbt language:swb language:tur language:dyo language:sin language:pbu language:ada language:pap language:qvh language:loz language:pan language:qva language:sme language:bax language:tuk language:hsb language:hus language:qvn language:ban language:cha language:zyb language:hin language:tat language:qxu language:gej language:quc language:mnw language:bho language:udu language:kha language:kbr language:tsz language:pau language:mkd language:shp language:ike language:lue language:tgl language:yap language:yua language:koi language:hrv language:emk language:tet language:ndo language:cbu language:vep language:cmn language:sag language:nym language:rus language:gjn language:guk language:kri language:ote language:lun language:vai language:bis language:arn language:tsn language:gle language:hak language:gkp language:ura language:tca language:xho language:wln language:amc language:mos language:lld language:bul language:qxn language:bcl language:ctd language:dip language:dag language:kek language:bre language:mri language:fin language:sah language:cym language:kan language:fao language:gsw language:sey language:bem language:bos language:bin language:chv language:tpi language:ami language:oaa language:lob language:ast language:nno language:sco language:khm language:pes language:pbb language:tam language:ibo language:sid language:plt language:guj language:hsn language:kin language:lug language:slr language:koo language:xsm language:jpn language:oki language:deu language:rar language:pcm language:hni language:vec language:gld language:sot language:crs language:fuv language:npi language:nya language:kea language:blt language:roh language:cbr language:chk language:kal language:mfq language:quh language:kor language:slv language:cof language:shk language:zul language:qwh language:fon language:mic language:prs language:mag language:bel language:iii language:mar language:dyu language:boa language:swe language:pis language:mlt language:amh language:umb language:cnr language:mai language:toj language:csw language:ina language:bba language:cbs language:kng language:oci language:pcd language:miq language:lat language:qvm language:wwa language:urd language:kmr language:ido language:gaa language:epo language:gaz language:cat language:hye language:cni language:suk language:gug language:gan language:cjk language:tzh language:zam language:ces language:cic language:mcf language:not language:kaa language:tso language:piu language:fat language:mzi language:snn language:tly language:eus language:nld language:nob language:wol language:hlt language:sna language:tiv language:ton language:hea language:skr language:lns language:rup language:cab language:glg language:yao language:nyn language:aii language:slk language:ukr language:kkh language:zdj language:amr language:yue language:crh language:hil license:cc0-1.0 size_categories:10K<n<100K format:csv modality:text library:datasets library:pandas library:mlcroissant library:polars arxiv:2309.13320 arxiv:2310.16248 region:us UDHR udhr language identification LID glot GlotLID