task_categories:token-classification language:nia language:avk language:bjn language:anp language:si language:bcc language:sma language:syl language:nso language:lez language:qug language:ks language:rmc language:khw language:szy language:kk language:uz language:sro language:tg language:jbo language:gld language:cu language:nb language:gom language:zgh language:iu language:th language:lv language:eo language:aln language:te language:nrm language:udm language:bcl language:ku language:rup language:ga language:es language:bm language:crh language:na language:yi language:mh language:grc language:lmo language:ff language:fi language:ty language:ay language:pi language:ii language:kea language:zh language:ce language:km language:vep language:ch language:br language:rw language:prg language:rue language:bxr language:hu language:kcg language:be language:tly language:sr language:dag language:rm language:ve language:nah language:hr language:dv language:mrj language:ko language:as language:ext language:liv language:ho language:fa language:pam language:en language:mnw language:abs language:mnc language:fy language:sq language:xal language:ia language:nv language:mdf language:pms language:av language:vec language:koi language:sco language:nl language:pfl language:loz language:xmf language:ng language:scn language:war language:ka language:da language:el language:nn language:cs language:li language:pdc language:no language:ki language:so language:krl language:rn language:nds language:lbe language:it language:gu language:mai language:mni language:sh language:frr language:uk language:qu language:am language:io language:ms language:sg language:mos language:pap language:ksh language:gn language:gor language:fon language:mus language:pnb language:nan language:sat language:kbp language:niu language:co language:fur language:glk language:lo language:om language:shi language:haw language:tw language:ab language:ro language:ota language:os language:ur language:kl language:cho language:hi language:jut language:ik language:ky language:sli language:sn language:lij language:ht language:pt language:wal language:ba language:gd language:ban language:tum language:fj language:arz language:dty language:ta language:my language:srq language:ts language:lb language:sl language:tok language:pa language:dsb language:aa language:kg language:lt language:shn language:rmy language:csb language:tyv language:bho language:sa language:cdo language:kn language:pl language:tet language:gag language:bqi language:got language:bg language:bs language:kus language:rif language:ml language:mn language:inh language:pcm language:smj language:sm language:mt language:ang language:lld language:ti language:arq language:cnh language:srn language:cr language:sc language:mg language:ak language:fr language:tt language:mhr language:kbd language:pag language:sk language:gv language:ee language:ss language:eu language:vi language:de language:bn language:ig language:mzn language:sw language:ny language:oc language:kw language:mk language:lad language:la language:ln language:st language:smn language:bug language:mo language:als language:gl language:ca language:lfn language:su language:tl language:kiu language:ne language:lus language:zu language:lg language:is language:hyw language:brh language:szl language:za language:agq language:to language:he language:vot language:sdc language:gsw language:vmw language:mwl language:bgn language:mi language:or language:ckb language:nqo language:arn language:chy language:dz language:awa language:chr language:stq language:id language:kr language:et language:ltg language:dtp language:hil language:sei language:skr language:sms language:myv language:yo language:ru language:ceb language:kaa language:gan language:mad language:sah language:sd language:frp language:ami language:bar language:bew language:ady language:bbc language:bo language:wa language:pih language:wuu language:tr language:bpy language:nap language:kab language:lrc language:mr language:rgn language:an language:krc language:min language:hak language:diq language:new language:ksw language:vmf language:wo language:yue language:ja language:sv language:zea language:jv language:ast language:ar language:ug language:fit language:ses language:tn language:eml language:mrh language:dga language:ilo language:arc language:fat language:az language:ie language:nov language:din language:kv language:krj language:ary language:hsb language:cv language:bi language:hz language:hy language:ase language:hsn language:atj language:vo language:jam language:lki language:cy language:ha language:sty language:hif language:ps language:ace language:tk language:gaa language:se language:alt language:af language:tpi language:xh language:lzh language:kri language:vro language:fo license:mit size_categories:100M<n<1B format:parquet modality:text library:datasets library:dask library:mlcroissant library:polars arxiv:2405.09496 region:us wikipedia