Regarding Test Sets for the GEM datasets
MEMBER
Hi @vyraun ! The test references for CommonGen are not publicly available: you can reach out to the original dataset authors if you would like to ask for them, but we will not be releasing them as part of GEM (March 31st was the release date for the test set inputs, references are incidentally released for some of the test sets but shouldn't really be used for benchmark submissions)

cc @sebastiangehrmann
2021-04-05T13:19:32Z
https://github.com/huggingface/datasets/issues/2166#issuecomment-813388816
https://api.github.com/repos/huggingface/datasets/issues/2166
MDEyOklzc3VlQ29tbWVudDgxMzM4ODgxNg==
https://api.github.com/repos/huggingface/datasets/issues/comments/813388816/reactions
2021-04-05T13:20:02Z
https://api.github.com/repos/huggingface/datasets/issues/comments/813388816
https://avatars.githubusercontent.com/u/10469459?v=4
https://api.github.com/users/yjernite/events{/privacy}
https://api.github.com/users/yjernite/followers
https://api.github.com/users/yjernite/following{/other_user}
https://api.github.com/users/yjernite/gists{/gist_id}
https://github.com/yjernite
yjernite
MDQ6VXNlcjEwNDY5NDU5
https://api.github.com/users/yjernite/orgs
https://api.github.com/users/yjernite/received_events
https://api.github.com/users/yjernite/repos
https://api.github.com/users/yjernite/starred{/owner}{/repo}
https://api.github.com/users/yjernite/subscriptions
User
https://api.github.com/users/yjernite
public
NONE
Oh okay, thanks @yjernite ! 
2021-04-06T08:13:08Z
https://github.com/huggingface/datasets/issues/2166#issuecomment-813923888
https://api.github.com/repos/huggingface/datasets/issues/2166
MDEyOklzc3VlQ29tbWVudDgxMzkyMzg4OA==
https://api.github.com/repos/huggingface/datasets/issues/comments/813923888/reactions
2021-04-06T08:13:08Z
https://api.github.com/repos/huggingface/datasets/issues/comments/813923888
https://avatars.githubusercontent.com/u/17217068?v=4
https://api.github.com/users/vyraun/events{/privacy}
https://api.github.com/users/vyraun/followers
https://api.github.com/users/vyraun/following{/other_user}
https://api.github.com/users/vyraun/gists{/gist_id}
https://github.com/vyraun
vyraun
MDQ6VXNlcjE3MjE3MDY4
https://api.github.com/users/vyraun/orgs
https://api.github.com/users/vyraun/received_events
https://api.github.com/users/vyraun/repos
https://api.github.com/users/vyraun/starred{/owner}{/repo}
https://api.github.com/users/vyraun/subscriptions
User
https://api.github.com/users/vyraun
public
@yjernite Hi, are the test sets for the GEM datasets scheduled to be [added soon](https://gem-benchmark.com/shared_task)? 

e.g.

```
from datasets import load_dataset
DATASET_NAME="common_gen"
data = load_dataset("gem", DATASET_NAME)
```

The test set doesn't have the target or references.

```
data['test'][0]
{'concept_set_id': 0, 'concepts': ['drill', 'field', 'run', 'team'], 'gem_id': 'common_gen-test-0', 'gem_parent_id': 'common_gen-test-0', 'references': [], 'target': ''}
```

