task_categories:translation task_categories:text-retrieval multilinguality:translation multilinguality:multilingual source_datasets:mafand source_datasets:wmt-news source_datasets:globalvoices language:am language:ar language:ay language:bm language:bbj language:bn language:bg language:ca language:cs language:ku language:da language:de language:el language:en language:et language:ee language:fil language:fi language:fr language:fon language:gu language:ha language:he language:hi language:hu language:ig language:id language:it language:ja language:kk language:km language:ko language:lv language:lt language:lg language:luo language:mk language:mos language:my language:nl language:ne language:or language:pa language:pcm language:fa language:pl language:pt language:mg language:ro language:ru language:es language:sr language:sq language:sw language:sv language:tet language:tn language:tr language:tw language:ur language:wo language:yo language:zh language:zu license:cc-by-nc-4.0 size_categories:1M<n<10M format:parquet modality:image modality:text library:datasets library:pandas library:mlcroissant library:polars arxiv:2406.12634 region:us news polynews-parallel mafand globalvoices wmtnews