OSCAR from Inria group
MEMBER
PR is already open here : #348 
The only thing remaining is to compute the metadata of each subdataset (one per language + shuffled/unshuffled).
As soon as #863 is merged we can start computing them. This will take a bit of time though
2020-11-18T14:43:20Z
https://github.com/huggingface/datasets/issues/866#issuecomment-729724866
https://api.github.com/repos/huggingface/datasets/issues/866
MDEyOklzc3VlQ29tbWVudDcyOTcyNDg2Ng==
https://api.github.com/repos/huggingface/datasets/issues/comments/729724866/reactions
2020-11-18T14:43:20Z
https://api.github.com/repos/huggingface/datasets/issues/comments/729724866
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
Grand, thanks for this!
2020-11-18T15:01:30Z
https://github.com/huggingface/datasets/issues/866#issuecomment-729736549
https://api.github.com/repos/huggingface/datasets/issues/866
MDEyOklzc3VlQ29tbWVudDcyOTczNjU0OQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/729736549/reactions
2020-11-18T15:01:30Z
https://api.github.com/repos/huggingface/datasets/issues/comments/729736549
https://avatars.githubusercontent.com/u/34098722?v=4
https://api.github.com/users/jchwenger/events{/privacy}
https://api.github.com/users/jchwenger/followers
https://api.github.com/users/jchwenger/following{/other_user}
https://api.github.com/users/jchwenger/gists{/gist_id}
https://github.com/jchwenger
jchwenger
MDQ6VXNlcjM0MDk4NzIy
https://api.github.com/users/jchwenger/orgs
https://api.github.com/users/jchwenger/received_events
https://api.github.com/users/jchwenger/repos
https://api.github.com/users/jchwenger/starred{/owner}{/repo}
https://api.github.com/users/jchwenger/subscriptions
User
https://api.github.com/users/jchwenger
public
## Adding a Dataset
- **Name:** *OSCAR* (Open Super-large Crawled ALMAnaCH coRpus), multilingual parsing of Common Crawl (separate crawls for many different languages), [here](https://oscar-corpus.com/).
- **Description:** *OSCAR or Open Super-large Crawled ALMAnaCH coRpus is a huge multilingual corpus obtained by language classification and filtering of the Common Crawl corpus using the goclassy architecture.*
- **Paper:** *[here](https://hal.inria.fr/hal-02148693)*
- **Data:** *[here](https://oscar-corpus.com/)*
- **Motivation:** *useful for unsupervised tasks in separate languages. In an ideal world, your team would be able to obtain the unshuffled version, that could be used to train GPT-2-like models (the shuffled version, I suppose, could be used for translation).*

I am aware that you do offer the "colossal" Common Crawl dataset already, but this has the advantage to be available in many subcorpora for different languages.
