Adding pseudo-labels to datasets
MEMBER
Nice ! :)
It's indeed the first time we have such contributions so we'll have to figure out the appropriate way to integrate them.
Could you add details on what they could be used for ?

2020-10-12T10:11:50Z
https://github.com/huggingface/datasets/issues/723#issuecomment-707025608
https://api.github.com/repos/huggingface/datasets/issues/723
MDEyOklzc3VlQ29tbWVudDcwNzAyNTYwOA==
https://api.github.com/repos/huggingface/datasets/issues/comments/707025608/reactions
2020-10-12T10:11:50Z
https://api.github.com/repos/huggingface/datasets/issues/comments/707025608
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
They can be used as training data for a smaller model.
2020-10-12T14:37:35Z
https://github.com/huggingface/datasets/issues/723#issuecomment-707160383
https://api.github.com/repos/huggingface/datasets/issues/723
MDEyOklzc3VlQ29tbWVudDcwNzE2MDM4Mw==
https://api.github.com/repos/huggingface/datasets/issues/comments/707160383/reactions
2020-10-12T14:37:35Z
https://api.github.com/repos/huggingface/datasets/issues/comments/707160383
https://avatars.githubusercontent.com/u/6045025?v=4
https://api.github.com/users/sshleifer/events{/privacy}
https://api.github.com/users/sshleifer/followers
https://api.github.com/users/sshleifer/following{/other_user}
https://api.github.com/users/sshleifer/gists{/gist_id}
https://github.com/sshleifer
sshleifer
MDQ6VXNlcjYwNDUwMjU=
https://api.github.com/users/sshleifer/orgs
https://api.github.com/users/sshleifer/received_events
https://api.github.com/users/sshleifer/repos
https://api.github.com/users/sshleifer/starred{/owner}{/repo}
https://api.github.com/users/sshleifer/subscriptions
User
https://api.github.com/users/sshleifer
public
MEMBER
Sounds just like a regular dataset to me then, no?
2020-10-13T07:50:46Z
https://github.com/huggingface/datasets/issues/723#issuecomment-707560158
https://api.github.com/repos/huggingface/datasets/issues/723
MDEyOklzc3VlQ29tbWVudDcwNzU2MDE1OA==
https://api.github.com/repos/huggingface/datasets/issues/comments/707560158/reactions
2020-10-13T07:50:46Z
https://api.github.com/repos/huggingface/datasets/issues/comments/707560158
https://avatars.githubusercontent.com/u/7353373?v=4
https://api.github.com/users/thomwolf/events{/privacy}
https://api.github.com/users/thomwolf/followers
https://api.github.com/users/thomwolf/following{/other_user}
https://api.github.com/users/thomwolf/gists{/gist_id}
https://github.com/thomwolf
thomwolf
MDQ6VXNlcjczNTMzNzM=
https://api.github.com/users/thomwolf/orgs
https://api.github.com/users/thomwolf/received_events
https://api.github.com/users/thomwolf/repos
https://api.github.com/users/thomwolf/starred{/owner}{/repo}
https://api.github.com/users/thomwolf/subscriptions
User
https://api.github.com/users/thomwolf
public
MEMBER
A new configuration for those datasets should do the job then.
Note that until now datasets like xsum only had one configuration. It means that users didn't have to specify the configuration name when loading the dataset. If we add new configs, users that update the lib will have to update their code to specify the default/standard configuration name (not the one with pseudo labels).
2020-10-13T07:57:50Z
https://github.com/huggingface/datasets/issues/723#issuecomment-707563907
https://api.github.com/repos/huggingface/datasets/issues/723
MDEyOklzc3VlQ29tbWVudDcwNzU2MzkwNw==
https://api.github.com/repos/huggingface/datasets/issues/comments/707563907/reactions
2020-10-13T07:57:50Z
https://api.github.com/repos/huggingface/datasets/issues/comments/707563907
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
Could also be a `user-namespace` dataset maybe?
2020-10-13T08:48:31Z
https://github.com/huggingface/datasets/issues/723#issuecomment-707593056
https://api.github.com/repos/huggingface/datasets/issues/723
MDEyOklzc3VlQ29tbWVudDcwNzU5MzA1Ng==
https://api.github.com/repos/huggingface/datasets/issues/comments/707593056/reactions
2020-10-13T08:48:38Z
https://api.github.com/repos/huggingface/datasets/issues/comments/707593056
https://avatars.githubusercontent.com/u/7353373?v=4
https://api.github.com/users/thomwolf/events{/privacy}
https://api.github.com/users/thomwolf/followers
https://api.github.com/users/thomwolf/following{/other_user}
https://api.github.com/users/thomwolf/gists{/gist_id}
https://github.com/thomwolf
thomwolf
MDQ6VXNlcjczNTMzNzM=
https://api.github.com/users/thomwolf/orgs
https://api.github.com/users/thomwolf/received_events
https://api.github.com/users/thomwolf/repos
https://api.github.com/users/thomwolf/starred{/owner}{/repo}
https://api.github.com/users/thomwolf/subscriptions
User
https://api.github.com/users/thomwolf
public
MEMBER
Oh yes why not. I'm more in favor of this actually since pseudo labels are things that users (not dataset authors in general) can compute by themselves and share with the community
2020-10-13T08:51:07Z
https://github.com/huggingface/datasets/issues/723#issuecomment-707594536
https://api.github.com/repos/huggingface/datasets/issues/723
MDEyOklzc3VlQ29tbWVudDcwNzU5NDUzNg==
https://api.github.com/repos/huggingface/datasets/issues/comments/707594536/reactions
2020-10-13T08:52:13Z
https://api.github.com/repos/huggingface/datasets/issues/comments/707594536
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
![image](https://user-images.githubusercontent.com/6045025/96045248-b528a380-0e3f-11eb-9124-bd55afa031bb.png)

I assume I should (for example) rename the xsum dir, change the URL, and put the modified dir somewhere in S3?
2020-10-14T21:08:03Z
https://github.com/huggingface/datasets/issues/723#issuecomment-708660859
https://api.github.com/repos/huggingface/datasets/issues/723
MDEyOklzc3VlQ29tbWVudDcwODY2MDg1OQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/708660859/reactions
2020-10-14T21:08:03Z
https://api.github.com/repos/huggingface/datasets/issues/comments/708660859
https://avatars.githubusercontent.com/u/6045025?v=4
https://api.github.com/users/sshleifer/events{/privacy}
https://api.github.com/users/sshleifer/followers
https://api.github.com/users/sshleifer/following{/other_user}
https://api.github.com/users/sshleifer/gists{/gist_id}
https://github.com/sshleifer
sshleifer
MDQ6VXNlcjYwNDUwMjU=
https://api.github.com/users/sshleifer/orgs
https://api.github.com/users/sshleifer/received_events
https://api.github.com/users/sshleifer/repos
https://api.github.com/users/sshleifer/starred{/owner}{/repo}
https://api.github.com/users/sshleifer/subscriptions
User
https://api.github.com/users/sshleifer
public
MEMBER
You can use the `datasets-cli` to upload the folder with your version of xsum with the pseudo labels.

```
datasets-cli upload_dataset path/to/xsum
```
2020-10-15T09:33:58Z
https://github.com/huggingface/datasets/issues/723#issuecomment-709041325
https://api.github.com/repos/huggingface/datasets/issues/723
MDEyOklzc3VlQ29tbWVudDcwOTA0MTMyNQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/709041325/reactions
2020-10-15T09:33:58Z
https://api.github.com/repos/huggingface/datasets/issues/comments/709041325
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
I recently [uploaded pseudo-labels](https://github.com/huggingface/transformers/blob/master/examples/seq2seq/precomputed_pseudo_labels.md) for CNN/DM, XSUM and WMT16-en-ro to s3, and thom mentioned I should add them to this repo.
Since pseudo-labels are just a large model's generations on an existing dataset, what is the right way to structure this contribution.
I read https://huggingface.co/docs/datasets/add_dataset.html, but it doesn't really cover this type of contribution.

I could, for example, make a new directory, `xsum_bart_pseudolabels` for each set of pseudolabels or add some sort of parametrization to `xsum.py`: https://github.com/huggingface/datasets/blob/5f4c6e830f603830117877b8990a0e65a2386aa6/datasets/xsum/xsum.py

What do you think @lhoestq ?


