task_categories:translation task_categories:text-generation task_categories:summarization language:ach language:ada language:adh language:aeb language:afr language:alz language:amh language:anu language:ara language:asm language:bem language:ben language:bod language:bxk language:cat language:cce language:ckb language:crk language:csw language:ctu language:dag language:dan language:deu language:dga language:din language:dje language:ell language:eng language:epo language:ewe language:fas language:fat language:fra language:ful language:gaa language:gjn language:guj language:gur language:guz language:gyn language:hat language:hau language:hbs language:hch language:her language:hin language:hun language:hus language:ind language:ita language:jam language:jpn language:kam language:kan language:kau language:kdj language:keo language:khm language:kik language:kin language:kln language:kmr language:kok language:koo language:kor language:kpz language:kqn language:kri language:kru language:ktz language:kua language:kwn language:laj language:lat language:lgg language:lin language:lit language:lko language:loz language:lsm language:luc language:lue language:lug language:lun language:luo language:lwg language:mal language:mar language:mas language:mat language:maz language:mer language:mfe language:mhi language:mhw language:miu language:mlg language:mmc language:mnw language:mqu language:msa language:mya language:myx language:naq language:nbl language:nch language:ndo language:nep language:nhe language:nhw language:nld language:nle language:nno language:nob language:nor language:nso language:nuj language:nya language:nyn language:nyu language:nzi language:ocu language:old language:ori language:orm language:pan language:pcm language:pmq language:pol language:por language:prs language:pus language:rki language:ron language:rus language:sag language:san language:saq language:sck language:sme language:som language:sot language:spa language:sqi language:srp language:ssw language:swa language:swe language:tam language:tel language:teo language:tet language:tgl language:tha language:tir language:toh language:toi language:tsc language:tsn language:tso language:ttj language:tum language:tur language:tuv language:twi language:ukr language:urd language:ven language:vie language:xho language:xog language:xsm language:yor language:yua language:yue language:zho language:zne language:zul license:cc size_categories:10K<n<100K format:csv modality:text library:datasets library:pandas library:mlcroissant library:polars arxiv:2309.13320 arxiv:2310.16248 region:us storybook book story language-identification nalibali machine-translation