[Dataset requests] New datasets for Text Classification
MEMBER
Pinging @mariamabarham as well
2020-07-08T12:18:13Z
https://github.com/huggingface/datasets/issues/353#issuecomment-655483264
https://api.github.com/repos/huggingface/datasets/issues/353
MDEyOklzc3VlQ29tbWVudDY1NTQ4MzI2NA==
https://api.github.com/repos/huggingface/datasets/issues/comments/655483264/reactions
2020-07-08T12:18:13Z
https://api.github.com/repos/huggingface/datasets/issues/comments/655483264
https://avatars.githubusercontent.com/u/7353373?v=4
https://api.github.com/users/thomwolf/events{/privacy}
https://api.github.com/users/thomwolf/followers
https://api.github.com/users/thomwolf/following{/other_user}
https://api.github.com/users/thomwolf/gists{/gist_id}
https://github.com/thomwolf
thomwolf
MDQ6VXNlcjczNTMzNzM=
https://api.github.com/users/thomwolf/orgs
https://api.github.com/users/thomwolf/received_events
https://api.github.com/users/thomwolf/repos
https://api.github.com/users/thomwolf/starred{/owner}{/repo}
https://api.github.com/users/thomwolf/subscriptions
User
https://api.github.com/users/thomwolf
public
CONTRIBUTOR
- `nlp` has MR! It's called `rotten_tomatoes`
- SST is part of GLUE, or is that just SST-2?
- `nlp` also has `ag_news`, a popular news classification dataset

I'd also like to see:
- the Yahoo Answers topic classification dataset
- the Kaggle Fake News classification dataset
2020-07-08T16:48:35Z
https://github.com/huggingface/datasets/issues/353#issuecomment-655634341
https://api.github.com/repos/huggingface/datasets/issues/353
MDEyOklzc3VlQ29tbWVudDY1NTYzNDM0MQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/655634341/reactions
2020-07-08T16:50:15Z
https://api.github.com/repos/huggingface/datasets/issues/comments/655634341
https://avatars.githubusercontent.com/u/13238952?v=4
https://api.github.com/users/jxmorris12/events{/privacy}
https://api.github.com/users/jxmorris12/followers
https://api.github.com/users/jxmorris12/following{/other_user}
https://api.github.com/users/jxmorris12/gists{/gist_id}
https://github.com/jxmorris12
jxmorris12
MDQ6VXNlcjEzMjM4OTUy
https://api.github.com/users/jxmorris12/orgs
https://api.github.com/users/jxmorris12/received_events
https://api.github.com/users/jxmorris12/repos
https://api.github.com/users/jxmorris12/starred{/owner}{/repo}
https://api.github.com/users/jxmorris12/subscriptions
User
https://api.github.com/users/jxmorris12
public
CONTRIBUTOR
Thanks @jxmorris12 for pointing this out. 

In glue we only have SST-2 maybe we can add separately SST-1.

2020-07-08T16:55:44Z
https://github.com/huggingface/datasets/issues/353#issuecomment-655637951
https://api.github.com/repos/huggingface/datasets/issues/353
MDEyOklzc3VlQ29tbWVudDY1NTYzNzk1MQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/655637951/reactions
2020-07-08T16:55:44Z
https://api.github.com/repos/huggingface/datasets/issues/comments/655637951
https://avatars.githubusercontent.com/u/38249783?v=4
https://api.github.com/users/mariamabarham/events{/privacy}
https://api.github.com/users/mariamabarham/followers
https://api.github.com/users/mariamabarham/following{/other_user}
https://api.github.com/users/mariamabarham/gists{/gist_id}
https://github.com/mariamabarham
mariamabarham
MDQ6VXNlcjM4MjQ5Nzgz
https://api.github.com/users/mariamabarham/orgs
https://api.github.com/users/mariamabarham/received_events
https://api.github.com/users/mariamabarham/repos
https://api.github.com/users/mariamabarham/starred{/owner}{/repo}
https://api.github.com/users/mariamabarham/subscriptions
User
https://api.github.com/users/mariamabarham
public
CONTRIBUTOR
This is the homepage for the Amazon dataset: https://www.kaggle.com/datafiniti/consumer-reviews-of-amazon-products

Is there an easy way to download kaggle datasets programmatically? If so, I can add this one!
2020-08-06T19:44:21Z
https://github.com/huggingface/datasets/issues/353#issuecomment-670156250
https://api.github.com/repos/huggingface/datasets/issues/353
MDEyOklzc3VlQ29tbWVudDY3MDE1NjI1MA==
https://api.github.com/repos/huggingface/datasets/issues/comments/670156250/reactions
2020-08-06T19:44:21Z
https://api.github.com/repos/huggingface/datasets/issues/comments/670156250
https://avatars.githubusercontent.com/u/13238952?v=4
https://api.github.com/users/jxmorris12/events{/privacy}
https://api.github.com/users/jxmorris12/followers
https://api.github.com/users/jxmorris12/following{/other_user}
https://api.github.com/users/jxmorris12/gists{/gist_id}
https://github.com/jxmorris12
jxmorris12
MDQ6VXNlcjEzMjM4OTUy
https://api.github.com/users/jxmorris12/orgs
https://api.github.com/users/jxmorris12/received_events
https://api.github.com/users/jxmorris12/repos
https://api.github.com/users/jxmorris12/starred{/owner}{/repo}
https://api.github.com/users/jxmorris12/subscriptions
User
https://api.github.com/users/jxmorris12
public
CONTRIBUTOR
Hi @jxmorris12 for now I think our `dl_manager` does not download from Kaggle.
@thomwolf , @lhoestq
2020-08-07T07:21:30Z
https://github.com/huggingface/datasets/issues/353#issuecomment-670371108
https://api.github.com/repos/huggingface/datasets/issues/353
MDEyOklzc3VlQ29tbWVudDY3MDM3MTEwOA==
https://api.github.com/repos/huggingface/datasets/issues/comments/670371108/reactions
2020-08-07T07:21:30Z
https://api.github.com/repos/huggingface/datasets/issues/comments/670371108
https://avatars.githubusercontent.com/u/38249783?v=4
https://api.github.com/users/mariamabarham/events{/privacy}
https://api.github.com/users/mariamabarham/followers
https://api.github.com/users/mariamabarham/following{/other_user}
https://api.github.com/users/mariamabarham/gists{/gist_id}
https://github.com/mariamabarham
mariamabarham
MDQ6VXNlcjM4MjQ5Nzgz
https://api.github.com/users/mariamabarham/orgs
https://api.github.com/users/mariamabarham/received_events
https://api.github.com/users/mariamabarham/repos
https://api.github.com/users/mariamabarham/starred{/owner}{/repo}
https://api.github.com/users/mariamabarham/subscriptions
User
https://api.github.com/users/mariamabarham
public
CONTRIBUTOR
Pretty sure the quora dataset is the same one I implemented here: https://github.com/huggingface/nlp/pull/366
2020-08-07T12:14:24Z
https://github.com/huggingface/datasets/issues/353#issuecomment-670487136
https://api.github.com/repos/huggingface/datasets/issues/353
MDEyOklzc3VlQ29tbWVudDY3MDQ4NzEzNg==
https://api.github.com/repos/huggingface/datasets/issues/comments/670487136/reactions
2020-08-07T12:14:24Z
https://api.github.com/repos/huggingface/datasets/issues/comments/670487136
https://avatars.githubusercontent.com/u/13795113?v=4
https://api.github.com/users/ghomasHudson/events{/privacy}
https://api.github.com/users/ghomasHudson/followers
https://api.github.com/users/ghomasHudson/following{/other_user}
https://api.github.com/users/ghomasHudson/gists{/gist_id}
https://github.com/ghomasHudson
ghomasHudson
MDQ6VXNlcjEzNzk1MTEz
https://api.github.com/users/ghomasHudson/orgs
https://api.github.com/users/ghomasHudson/received_events
https://api.github.com/users/ghomasHudson/repos
https://api.github.com/users/ghomasHudson/starred{/owner}{/repo}
https://api.github.com/users/ghomasHudson/subscriptions
User
https://api.github.com/users/ghomasHudson
public
NONE
Great list. Any idea if Amazon Reviews has been added?

- ~40 GB of text (sadly no emoji)
- popular MLM pre-training dataset before bigger datasets like WebText https://arxiv.org/abs/1808.01371
- turns out that binarizing the 1-5 star rating leads to great Pos/Neg/Neutral dataset, T5 paper claims to get very high accuracy (98%!) on this with small amount of finetuning https://arxiv.org/abs/2004.14546

Apologies if it's been included (great to see where) and if not, it's one of the better medium/large NLP dataset for semi-supervised learning, albeit a bit out of date. 

Thanks!! 

cc @sshleifer 
2020-09-17T03:17:09Z
https://github.com/huggingface/datasets/issues/353#issuecomment-693783420
https://api.github.com/repos/huggingface/datasets/issues/353
MDEyOklzc3VlQ29tbWVudDY5Mzc4MzQyMA==
https://api.github.com/repos/huggingface/datasets/issues/comments/693783420/reactions
2020-09-17T03:17:09Z
https://api.github.com/repos/huggingface/datasets/issues/comments/693783420
https://avatars.githubusercontent.com/u/1473764?v=4
https://api.github.com/users/moscow25/events{/privacy}
https://api.github.com/users/moscow25/followers
https://api.github.com/users/moscow25/following{/other_user}
https://api.github.com/users/moscow25/gists{/gist_id}
https://github.com/moscow25
moscow25
MDQ6VXNlcjE0NzM3NjQ=
https://api.github.com/users/moscow25/orgs
https://api.github.com/users/moscow25/received_events
https://api.github.com/users/moscow25/repos
https://api.github.com/users/moscow25/starred{/owner}{/repo}
https://api.github.com/users/moscow25/subscriptions
User
https://api.github.com/users/moscow25
public
NONE
On the Amazon Reviews dataset, the original UCSD website has noted these are now updated to include product reviews through 2018 -- actually quite recent compared to many other datasets. Almost certainly the largest NLP dataset out there with labels!
https://jmcauley.ucsd.edu/data/amazon/ 

Any chance someone has time to onboard this dataset in a HF way?

cc @sshleifer 
2020-10-20T03:41:23Z
https://github.com/huggingface/datasets/issues/353#issuecomment-712568673
https://api.github.com/repos/huggingface/datasets/issues/353
MDEyOklzc3VlQ29tbWVudDcxMjU2ODY3Mw==
https://api.github.com/repos/huggingface/datasets/issues/comments/712568673/reactions
2020-10-20T03:41:23Z
https://api.github.com/repos/huggingface/datasets/issues/comments/712568673
https://avatars.githubusercontent.com/u/1473764?v=4
https://api.github.com/users/moscow25/events{/privacy}
https://api.github.com/users/moscow25/followers
https://api.github.com/users/moscow25/following{/other_user}
https://api.github.com/users/moscow25/gists{/gist_id}
https://github.com/moscow25
moscow25
MDQ6VXNlcjE0NzM3NjQ=
https://api.github.com/users/moscow25/orgs
https://api.github.com/users/moscow25/received_events
https://api.github.com/users/moscow25/repos
https://api.github.com/users/moscow25/starred{/owner}{/repo}
https://api.github.com/users/moscow25/subscriptions
User
https://api.github.com/users/moscow25
public
NONE
@albertvillanova How up to date is this issue? I see that some of these datasets are now on huggingface but have not been checked off the list
2024-02-07T20:07:14Z
https://github.com/huggingface/datasets/issues/353#issuecomment-1932782602
https://api.github.com/repos/huggingface/datasets/issues/353
IC_kwDODunzps5zM-wK
https://api.github.com/repos/huggingface/datasets/issues/comments/1932782602/reactions
2024-02-07T20:07:14Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1932782602
https://avatars.githubusercontent.com/u/93222362?v=4
https://api.github.com/users/johneckberg/events{/privacy}
https://api.github.com/users/johneckberg/followers
https://api.github.com/users/johneckberg/following{/other_user}
https://api.github.com/users/johneckberg/gists{/gist_id}
https://github.com/johneckberg
johneckberg
U_kgDOBY512g
https://api.github.com/users/johneckberg/orgs
https://api.github.com/users/johneckberg/received_events
https://api.github.com/users/johneckberg/repos
https://api.github.com/users/johneckberg/starred{/owner}{/repo}
https://api.github.com/users/johneckberg/subscriptions
User
https://api.github.com/users/johneckberg
public
NONE
The OHSUMED dataset is already available as a community dataset at [community-datasets/ohsumed](https://huggingface.co/datasets/community-datasets/ohsumed).

The community version includes:

- Full dataset with 348,566 references from MEDLINE
- Train/test splits (54,709 / 293,855 examples)
- All original fields (title, abstract, MeSH terms, author, source, publication type)

Given this existing implementation, we might want to Mark OHSUMED as completed in the issue list.
2025-04-05T02:59:38Z
https://github.com/huggingface/datasets/issues/353#issuecomment-2780161321
https://api.github.com/repos/huggingface/datasets/issues/353
IC_kwDODunzps6lteUp
https://api.github.com/repos/huggingface/datasets/issues/comments/2780161321/reactions
2025-04-05T02:59:38Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2780161321
https://avatars.githubusercontent.com/u/37915995?v=4
https://api.github.com/users/Yasserelhaddar/events{/privacy}
https://api.github.com/users/Yasserelhaddar/followers
https://api.github.com/users/Yasserelhaddar/following{/other_user}
https://api.github.com/users/Yasserelhaddar/gists{/gist_id}
https://github.com/Yasserelhaddar
Yasserelhaddar
MDQ6VXNlcjM3OTE1OTk1
https://api.github.com/users/Yasserelhaddar/orgs
https://api.github.com/users/Yasserelhaddar/received_events
https://api.github.com/users/Yasserelhaddar/repos
https://api.github.com/users/Yasserelhaddar/starred{/owner}{/repo}
https://api.github.com/users/Yasserelhaddar/subscriptions
User
https://api.github.com/users/Yasserelhaddar
public
NONE
I notice that several datasets from this list are already available:

1. OHSUMED: Available as [community-datasets/ohsumed](https://huggingface.co/datasets/community-datasets/ohsumed)
2. EUR-Lex: Available in two versions:
   - [coastalcph/multi_eurlex](https://huggingface.co/datasets/coastalcph/multi_eurlex)
   - [NLP-AUEB/eurlex](https://huggingface.co/datasets/NLP-AUEB/eurlex)
3. PubMed: Available as [ncbi/pubmed](https://huggingface.co/datasets/ncbi/pubmed)

Is this issue not updated, or am I missing something? The only dataset from the remaining list that seems to truly need addition is the MPQA dataset.
2025-04-05T03:06:29Z
https://github.com/huggingface/datasets/issues/353#issuecomment-2780164496
https://api.github.com/repos/huggingface/datasets/issues/353
IC_kwDODunzps6ltfGQ
https://api.github.com/repos/huggingface/datasets/issues/comments/2780164496/reactions
2025-04-05T03:06:29Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2780164496
https://avatars.githubusercontent.com/u/37915995?v=4
https://api.github.com/users/Yasserelhaddar/events{/privacy}
https://api.github.com/users/Yasserelhaddar/followers
https://api.github.com/users/Yasserelhaddar/following{/other_user}
https://api.github.com/users/Yasserelhaddar/gists{/gist_id}
https://github.com/Yasserelhaddar
Yasserelhaddar
MDQ6VXNlcjM3OTE1OTk1
https://api.github.com/users/Yasserelhaddar/orgs
https://api.github.com/users/Yasserelhaddar/received_events
https://api.github.com/users/Yasserelhaddar/repos
https://api.github.com/users/Yasserelhaddar/starred{/owner}{/repo}
https://api.github.com/users/Yasserelhaddar/subscriptions
User
https://api.github.com/users/Yasserelhaddar
public
CONTRIBUTOR
MPQA is definitely on the hub: https://huggingface.co/datasets/jxm/mpqa
2025-04-05T09:28:14Z
https://github.com/huggingface/datasets/issues/353#issuecomment-2780629980
https://api.github.com/repos/huggingface/datasets/issues/353
IC_kwDODunzps6lvQvc
https://api.github.com/repos/huggingface/datasets/issues/comments/2780629980/reactions
2025-04-05T09:28:14Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2780629980
https://avatars.githubusercontent.com/u/13238952?v=4
https://api.github.com/users/jxmorris12/events{/privacy}
https://api.github.com/users/jxmorris12/followers
https://api.github.com/users/jxmorris12/following{/other_user}
https://api.github.com/users/jxmorris12/gists{/gist_id}
https://github.com/jxmorris12
jxmorris12
MDQ6VXNlcjEzMjM4OTUy
https://api.github.com/users/jxmorris12/orgs
https://api.github.com/users/jxmorris12/received_events
https://api.github.com/users/jxmorris12/repos
https://api.github.com/users/jxmorris12/starred{/owner}{/repo}
https://api.github.com/users/jxmorris12/subscriptions
User
https://api.github.com/users/jxmorris12
public
We are missing a few datasets for Text Classification which is an important field.

Namely, it would be really nice to add:
- [x] TREC-6 dataset (see here for instance: https://pytorchnlp.readthedocs.io/en/latest/source/torchnlp.datasets.html#torchnlp.datasets.trec_dataset)  **[done]**
  - #386
- [x] Yelp-5
  - #1315
- [x] Movie review (Movie Review (MR) dataset [156]) **[done (same as rotten_tomatoes)]**
- [x] SST (Stanford Sentiment Treebank) **[include in glue]**
  - #1934
- [ ] Multi-Perspective Question Answering (MPQA) dataset **[require authentication (indeed manual download)]**
- [x] Amazon. This is a popular corpus of product reviews collected from the Amazon website [159]. It contains labels for both binary classification and multi-class (5-class) classification
  - #791
  - #1389
- [x] 20 Newsgroups. The 20 Newsgroups dataset  **[done]**
  - #410 
- [x] Sogou News dataset **[done]**
  - #450
- [x] Reuters news. The Reuters-21578 dataset [165] **[done]**
  - #471 
- [x] DBpedia. The DBpedia dataset [170]
  - #1116
- [ ] Ohsumed. The Ohsumed collection [171] is a subset of the MEDLINE database
- [ ] EUR-Lex. The EUR-Lex dataset
- [x] WOS. The Web Of Science (WOS) dataset **[done]**
  - #424
- [ ] PubMed. PubMed [173]
- [x] TREC-QA: TREC-6 + TREC-50
  - See above: TREC-6 dataset
- [x] Quora. The Quora dataset [180]
  - #366

All these datasets are cited in https://arxiv.org/abs/2004.03705