Adding CC-100: Monolingual Datasets from Web Crawl Data
MEMBER
cc @aconneau ;) 
2020-10-28T18:26:15Z
https://github.com/huggingface/datasets/issues/773#issuecomment-718124723
https://api.github.com/repos/huggingface/datasets/issues/773
MDEyOklzc3VlQ29tbWVudDcxODEyNDcyMw==
https://api.github.com/repos/huggingface/datasets/issues/comments/718124723/reactions
2020-10-28T18:26:15Z
https://api.github.com/repos/huggingface/datasets/issues/comments/718124723
https://avatars.githubusercontent.com/u/10469459?v=4
https://api.github.com/users/yjernite/events{/privacy}
https://api.github.com/users/yjernite/followers
https://api.github.com/users/yjernite/following{/other_user}
https://api.github.com/users/yjernite/gists{/gist_id}
https://github.com/yjernite
yjernite
MDQ6VXNlcjEwNDY5NDU5
https://api.github.com/users/yjernite/orgs
https://api.github.com/users/yjernite/received_events
https://api.github.com/users/yjernite/repos
https://api.github.com/users/yjernite/starred{/owner}{/repo}
https://api.github.com/users/yjernite/subscriptions
User
https://api.github.com/users/yjernite
public
CONTRIBUTOR
These dataset files are no longer available.  https://data.statmt.org/cc-100/ files provided in this link are no longer available. Can anybody fix that issue?
@abhishekkrthakur @yjernite 
2022-01-14T19:38:01Z
https://github.com/huggingface/datasets/issues/773#issuecomment-1013422678
https://api.github.com/repos/huggingface/datasets/issues/773
IC_kwDODunzps48Z5pW
https://api.github.com/repos/huggingface/datasets/issues/comments/1013422678/reactions
2022-01-14T19:38:01Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1013422678
https://avatars.githubusercontent.com/u/55232459?v=4
https://api.github.com/users/AnzorGozalishvili/events{/privacy}
https://api.github.com/users/AnzorGozalishvili/followers
https://api.github.com/users/AnzorGozalishvili/following{/other_user}
https://api.github.com/users/AnzorGozalishvili/gists{/gist_id}
https://github.com/AnzorGozalishvili
AnzorGozalishvili
MDQ6VXNlcjU1MjMyNDU5
https://api.github.com/users/AnzorGozalishvili/orgs
https://api.github.com/users/AnzorGozalishvili/received_events
https://api.github.com/users/AnzorGozalishvili/repos
https://api.github.com/users/AnzorGozalishvili/starred{/owner}{/repo}
https://api.github.com/users/AnzorGozalishvili/subscriptions
User
https://api.github.com/users/AnzorGozalishvili
public
MEMBER
Hi ! Can you open an issue to report this problem ? This will help keep track of the fix :)
2022-01-24T16:02:59Z
https://github.com/huggingface/datasets/issues/773#issuecomment-1020257935
https://api.github.com/repos/huggingface/datasets/issues/773
IC_kwDODunzps48z-aP
https://api.github.com/repos/huggingface/datasets/issues/comments/1020257935/reactions
2022-01-24T16:02:59Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1020257935
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Ok
2022-01-26T13:22:54Z
https://github.com/huggingface/datasets/issues/773#issuecomment-1022192869
https://api.github.com/repos/huggingface/datasets/issues/773
IC_kwDODunzps487Wzl
https://api.github.com/repos/huggingface/datasets/issues/comments/1022192869/reactions
2022-01-26T13:22:54Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1022192869
https://avatars.githubusercontent.com/u/55232459?v=4
https://api.github.com/users/AnzorGozalishvili/events{/privacy}
https://api.github.com/users/AnzorGozalishvili/followers
https://api.github.com/users/AnzorGozalishvili/following{/other_user}
https://api.github.com/users/AnzorGozalishvili/gists{/gist_id}
https://github.com/AnzorGozalishvili
AnzorGozalishvili
MDQ6VXNlcjU1MjMyNDU5
https://api.github.com/users/AnzorGozalishvili/orgs
https://api.github.com/users/AnzorGozalishvili/received_events
https://api.github.com/users/AnzorGozalishvili/repos
https://api.github.com/users/AnzorGozalishvili/starred{/owner}{/repo}
https://api.github.com/users/AnzorGozalishvili/subscriptions
User
https://api.github.com/users/AnzorGozalishvili
public
## Adding a Dataset
- **Name:** CC-100: Monolingual Datasets from Web Crawl Data
- **Description:** https://twitter.com/alex_conneau/status/1321507120848625665
- **Paper:** https://arxiv.org/abs/1911.02116
- **Data:** http://data.statmt.org/cc-100/
- **Motivation:** A large scale multi-lingual language modeling dataset. Text is de-duplicated and filtered by how "Wikipedia-like" it is, hopefully helping avoid some of the worst parts of the common crawl.

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).
