Add FewRel Dataset
CONTRIBUTOR
Hi @lhoestq,

Sorry for the late response. What do you mean when you say "adding names to default config"? Should I handle "pid2name" in the same config as "default"?
2021-02-18T14:00:16Z
https://github.com/huggingface/datasets/pull/1823#issuecomment-781363020
https://api.github.com/repos/huggingface/datasets/issues/1823
MDEyOklzc3VlQ29tbWVudDc4MTM2MzAyMA==
https://api.github.com/repos/huggingface/datasets/issues/comments/781363020/reactions
2021-02-18T14:00:56Z
https://api.github.com/repos/huggingface/datasets/issues/comments/781363020
https://avatars.githubusercontent.com/u/29076344?v=4
https://api.github.com/users/gchhablani/events{/privacy}
https://api.github.com/users/gchhablani/followers
https://api.github.com/users/gchhablani/following{/other_user}
https://api.github.com/users/gchhablani/gists{/gist_id}
https://github.com/gchhablani
gchhablani
MDQ6VXNlcjI5MDc2MzQ0
https://api.github.com/users/gchhablani/orgs
https://api.github.com/users/gchhablani/received_events
https://api.github.com/users/gchhablani/repos
https://api.github.com/users/gchhablani/starred{/owner}{/repo}
https://api.github.com/users/gchhablani/subscriptions
User
https://api.github.com/users/gchhablani
public
MEMBER
Yes I was thinking of having the pid2name field available in the default configuration (and therefore only have one config). What do you think ?
2021-02-18T16:31:58Z
https://github.com/huggingface/datasets/pull/1823#issuecomment-781469905
https://api.github.com/repos/huggingface/datasets/issues/1823
MDEyOklzc3VlQ29tbWVudDc4MTQ2OTkwNQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/781469905/reactions
2021-02-18T16:31:58Z
https://api.github.com/repos/huggingface/datasets/issues/comments/781469905
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Hi @lhoestq,

Sorry again, the last couple of weeks were a bit busy for me. I am wondering how do you want me to achieve that. Using a custom BuilderConfig which takes in whether it is the regular data or "pid2name"? "pid2name" is only useful for "train_wiki", "val_nyt" and "val_wiki". So, based on my understanding, it would look like this:

```python
wiki_data = load_dataset('few_rel','train_wiki')
id2name = load_dataset('few_rel','pid2name')
```
and this will be handled in the multiple configs.


A better alternative could be providing name of the relationship in only "train_wiki", "val_nyt" and "val_wiki" as an extra feature in the dataset, and doing away with "pid2name" entirely. I'll only download pid2name if any of those datasets are requested, and then during generation I'll return the list with the dataset under "names" feature. How does this sound?

EDIT:
There is one issue with the second approach, the entire pid2name is saved with all three datasets - "train_wiki", "val_nyt" and "val_wiki" ([see code below](https://github.com/huggingface/datasets/pull/1823#issuecomment-786402026)). In dummy data, I can address this by manually editing the pid2name to contain only a few id-name pairs, those matching with the examples in the corresponding example file. But this seems to be inefficient for the entire dataset - storing the same file in multiple places.
2021-02-26T01:16:02Z
https://github.com/huggingface/datasets/pull/1823#issuecomment-786340757
https://api.github.com/repos/huggingface/datasets/issues/1823
MDEyOklzc3VlQ29tbWVudDc4NjM0MDc1Nw==
https://api.github.com/repos/huggingface/datasets/issues/comments/786340757/reactions
2021-02-26T04:45:07Z
https://api.github.com/repos/huggingface/datasets/issues/comments/786340757
https://avatars.githubusercontent.com/u/29076344?v=4
https://api.github.com/users/gchhablani/events{/privacy}
https://api.github.com/users/gchhablani/followers
https://api.github.com/users/gchhablani/following{/other_user}
https://api.github.com/users/gchhablani/gists{/gist_id}
https://github.com/gchhablani
gchhablani
MDQ6VXNlcjI5MDc2MzQ0
https://api.github.com/users/gchhablani/orgs
https://api.github.com/users/gchhablani/received_events
https://api.github.com/users/gchhablani/repos
https://api.github.com/users/gchhablani/starred{/owner}{/repo}
https://api.github.com/users/gchhablani/subscriptions
User
https://api.github.com/users/gchhablani
public
CONTRIBUTOR
Okay, I apologize, I guess I finally understand what is required.

Basically, using:

```python
few_rel = load_dataset('few_rel')
```
should give all the files. This seems difficult since "pid2name" has a different format. Any suggestions on this?
2021-02-26T04:40:38Z
https://github.com/huggingface/datasets/pull/1823#issuecomment-786406389
https://api.github.com/repos/huggingface/datasets/issues/1823
MDEyOklzc3VlQ29tbWVudDc4NjQwNjM4OQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/786406389/reactions
2021-02-26T04:42:57Z
https://api.github.com/repos/huggingface/datasets/issues/comments/786406389
https://avatars.githubusercontent.com/u/29076344?v=4
https://api.github.com/users/gchhablani/events{/privacy}
https://api.github.com/users/gchhablani/followers
https://api.github.com/users/gchhablani/following{/other_user}
https://api.github.com/users/gchhablani/gists{/gist_id}
https://github.com/gchhablani
gchhablani
MDQ6VXNlcjI5MDc2MzQ0
https://api.github.com/users/gchhablani/orgs
https://api.github.com/users/gchhablani/received_events
https://api.github.com/users/gchhablani/repos
https://api.github.com/users/gchhablani/starred{/owner}{/repo}
https://api.github.com/users/gchhablani/subscriptions
User
https://api.github.com/users/gchhablani
public
MEMBER
Yes that's it, sorry if that wasn't clear !
2021-02-26T10:52:52Z
https://github.com/huggingface/datasets/pull/1823#issuecomment-786571786
https://api.github.com/repos/huggingface/datasets/issues/1823
MDEyOklzc3VlQ29tbWVudDc4NjU3MTc4Ng==
https://api.github.com/repos/huggingface/datasets/issues/comments/786571786/reactions
2021-02-26T10:52:52Z
https://api.github.com/repos/huggingface/datasets/issues/comments/786571786
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Hi @lhoestq,

Since pid2name has different features from the rest of the files, how will I add them to the same config?

Do we want to exclude pid2name totally and add "names" to every example?
2021-02-26T10:55:04Z
https://github.com/huggingface/datasets/pull/1823#issuecomment-786572970
https://api.github.com/repos/huggingface/datasets/issues/1823
MDEyOklzc3VlQ29tbWVudDc4NjU3Mjk3MA==
https://api.github.com/repos/huggingface/datasets/issues/comments/786572970/reactions
2021-02-26T10:56:32Z
https://api.github.com/repos/huggingface/datasets/issues/comments/786572970
https://avatars.githubusercontent.com/u/29076344?v=4
https://api.github.com/users/gchhablani/events{/privacy}
https://api.github.com/users/gchhablani/followers
https://api.github.com/users/gchhablani/following{/other_user}
https://api.github.com/users/gchhablani/gists{/gist_id}
https://github.com/gchhablani
gchhablani
MDQ6VXNlcjI5MDc2MzQ0
https://api.github.com/users/gchhablani/orgs
https://api.github.com/users/gchhablani/received_events
https://api.github.com/users/gchhablani/repos
https://api.github.com/users/gchhablani/starred{/owner}{/repo}
https://api.github.com/users/gchhablani/subscriptions
User
https://api.github.com/users/gchhablani
public
MEMBER
If I understand correctly each sample in the "default" config has one relation, and each relation has corresponding names in pid2name.
Would it be possible to also include the names in the "default" configuration for each sample ? The names of one sample can be retrieved using the relation id no ?
2021-02-26T10:56:51Z
https://github.com/huggingface/datasets/pull/1823#issuecomment-786573904
https://api.github.com/repos/huggingface/datasets/issues/1823
MDEyOklzc3VlQ29tbWVudDc4NjU3MzkwNA==
https://api.github.com/repos/huggingface/datasets/issues/comments/786573904/reactions
2021-02-26T10:56:51Z
https://api.github.com/repos/huggingface/datasets/issues/comments/786573904
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Yes, that can be done. But for some files, the name is already given instead of ID. Only "train_wiki", "val_wiki", "val_nyc" have IDs. For others, I can set the names equal to a list of key.
2021-02-26T10:58:45Z
https://github.com/huggingface/datasets/pull/1823#issuecomment-786574928
https://api.github.com/repos/huggingface/datasets/issues/1823
MDEyOklzc3VlQ29tbWVudDc4NjU3NDkyOA==
https://api.github.com/repos/huggingface/datasets/issues/comments/786574928/reactions
2021-02-26T11:00:41Z
https://api.github.com/repos/huggingface/datasets/issues/comments/786574928
https://avatars.githubusercontent.com/u/29076344?v=4
https://api.github.com/users/gchhablani/events{/privacy}
https://api.github.com/users/gchhablani/followers
https://api.github.com/users/gchhablani/following{/other_user}
https://api.github.com/users/gchhablani/gists{/gist_id}
https://github.com/gchhablani
gchhablani
MDQ6VXNlcjI5MDc2MzQ0
https://api.github.com/users/gchhablani/orgs
https://api.github.com/users/gchhablani/received_events
https://api.github.com/users/gchhablani/repos
https://api.github.com/users/gchhablani/starred{/owner}{/repo}
https://api.github.com/users/gchhablani/subscriptions
User
https://api.github.com/users/gchhablani
public
MEMBER
I think that's fine as long as we mention this processing explicitly in the dataset card.
2021-02-26T11:02:58Z
https://github.com/huggingface/datasets/pull/1823#issuecomment-786577313
https://api.github.com/repos/huggingface/datasets/issues/1823
MDEyOklzc3VlQ29tbWVudDc4NjU3NzMxMw==
https://api.github.com/repos/huggingface/datasets/issues/comments/786577313/reactions
2021-02-26T11:02:58Z
https://api.github.com/repos/huggingface/datasets/issues/comments/786577313
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Hi @lhoestq,

I have added the changes. Please let me know in case of any remaining issues.

Thanks,
Gunjan
2021-02-26T19:15:37Z
https://github.com/huggingface/datasets/pull/1823#issuecomment-786842178
https://api.github.com/repos/huggingface/datasets/issues/1823
MDEyOklzc3VlQ29tbWVudDc4Njg0MjE3OA==
https://api.github.com/repos/huggingface/datasets/issues/comments/786842178/reactions
2021-02-26T19:18:51Z
https://api.github.com/repos/huggingface/datasets/issues/comments/786842178
https://avatars.githubusercontent.com/u/29076344?v=4
https://api.github.com/users/gchhablani/events{/privacy}
https://api.github.com/users/gchhablani/followers
https://api.github.com/users/gchhablani/following{/other_user}
https://api.github.com/users/gchhablani/gists{/gist_id}
https://github.com/gchhablani
gchhablani
MDQ6VXNlcjI5MDc2MzQ0
https://api.github.com/users/gchhablani/orgs
https://api.github.com/users/gchhablani/received_events
https://api.github.com/users/gchhablani/repos
https://api.github.com/users/gchhablani/starred{/owner}{/repo}
https://api.github.com/users/gchhablani/subscriptions
User
https://api.github.com/users/gchhablani
public
CONTRIBUTOR
Hi @lhoestq,

Thanks for fixing it and approving :)
2021-03-01T11:56:20Z
https://github.com/huggingface/datasets/pull/1823#issuecomment-787892288
https://api.github.com/repos/huggingface/datasets/issues/1823
MDEyOklzc3VlQ29tbWVudDc4Nzg5MjI4OA==
https://api.github.com/repos/huggingface/datasets/issues/comments/787892288/reactions
2021-03-01T11:56:20Z
https://api.github.com/repos/huggingface/datasets/issues/comments/787892288
https://avatars.githubusercontent.com/u/29076344?v=4
https://api.github.com/users/gchhablani/events{/privacy}
https://api.github.com/users/gchhablani/followers
https://api.github.com/users/gchhablani/following{/other_user}
https://api.github.com/users/gchhablani/gists{/gist_id}
https://github.com/gchhablani
gchhablani
MDQ6VXNlcjI5MDc2MzQ0
https://api.github.com/users/gchhablani/orgs
https://api.github.com/users/gchhablani/received_events
https://api.github.com/users/gchhablani/repos
https://api.github.com/users/gchhablani/starred{/owner}{/repo}
https://api.github.com/users/gchhablani/subscriptions
User
https://api.github.com/users/gchhablani
public
Hi,

This PR closes this [Card](https://github.com/huggingface/datasets/projects/1#card-53285184) and Issue #1757.

I wasn't sure how to add `pid2name` along with the dataset so I added it as a separate configuration. For each (head, tail, tokens) triplet, I have created one example. I have added the dictionary key as `"relation"` in the dataset. Additionally, for `pubmed_unsupervised`, I kept `"relation":""` in the dictionary.

Please recommend better alternatives, if any.

Thanks,
Gunjan