task_categories:translation language:ak language:am language:ar language:awa language:azj language:bm language:ban language:be language:bem language:bn language:bho language:bjn language:bug language:bg language:ca language:ceb language:cs language:cjk language:ckb language:crh language:da language:de language:dik language:dyu language:el language:en language:eo language:et language:ee language:fo language:fj language:fi language:fon language:fr language:fur language:ff language:gaz language:gd language:ga language:gl language:gn language:gu language:ht language:ha language:he language:hi language:hne language:hr language:hu language:hy language:ig language:ilo language:id language:is language:it language:jv language:ja language:kab language:kac language:kam language:kn language:ks language:ka language:kk language:kbp language:kea language:mn language:km language:ki language:rw language:ky language:kmb language:kmr language:kr language:kg language:ko language:lo language:lij language:li language:ln language:lt language:lmo language:ltg language:lb language:lua language:lg language:luo language:lus language:lv language:mag language:mai language:ml language:mr language:min language:mk language:mt language:mni language:mos language:mi language:my language:nl language:nb language:ne language:nso language:nus language:ny language:oc language:ory language:pag language:pa language:pap language:pbt language:fa language:plt language:pl language:pt language:prs language:qu language:ro language:rn language:ru language:sg language:sa language:sat language:scn language:shn language:si language:sk language:sl language:sm language:sn language:sd language:so language:st language:es language:sc language:sr language:ss language:su language:sv language:sw language:szl language:ta language:taq language:tt language:te language:tg language:tl language:ti language:tpi language:tn language:ts language:tk language:tum language:tr language:tw language:tzm language:ug language:uk language:umb language:ur language:uz language:vec language:vi language:war language:wo language:xh language:yi language:yo language:zh language:ms language:zu license:odc-by size_categories:1M<n<10M format:parquet modality:tabular modality:text library:datasets library:dask library:mlcroissant library:polars arxiv:2207.04672 arxiv:2308.11596 region:us