CompGuessWhat?! dataset 
CONTRIBUTOR
Hi @aleSuglia, thanks for this great PR. Indeed you can have both datasets in one file. You need to add a config class which will allows you to specify the different subdataset names and then  you will be able to load them as follow.
nlp.load_dataset("compguesswhat", "compguesswhat-gameplay") 
nlp.load_dataset("compguesswhat", "compguesswhat-zs-gameplay").

Maybe you can refer to this file https://github.com/huggingface/nlp/blob/master/datasets/discofuse/discofuse.py
2020-06-05T08:07:26Z
https://github.com/huggingface/datasets/pull/236#issuecomment-639326552
https://api.github.com/repos/huggingface/datasets/issues/236
MDEyOklzc3VlQ29tbWVudDYzOTMyNjU1Mg==
https://api.github.com/repos/huggingface/datasets/issues/comments/639326552/reactions
2020-06-05T08:07:26Z
https://api.github.com/repos/huggingface/datasets/issues/comments/639326552
https://avatars.githubusercontent.com/u/38249783?v=4
https://api.github.com/users/mariamabarham/events{/privacy}
https://api.github.com/users/mariamabarham/followers
https://api.github.com/users/mariamabarham/following{/other_user}
https://api.github.com/users/mariamabarham/gists{/gist_id}
https://github.com/mariamabarham
mariamabarham
MDQ6VXNlcjM4MjQ5Nzgz
https://api.github.com/users/mariamabarham/orgs
https://api.github.com/users/mariamabarham/received_events
https://api.github.com/users/mariamabarham/repos
https://api.github.com/users/mariamabarham/starred{/owner}{/repo}
https://api.github.com/users/mariamabarham/subscriptions
User
https://api.github.com/users/mariamabarham
public
CONTRIBUTOR
@mariamabarham Thanks for your suggestions. I've followed your advice and integrated the additional dataset using another `DatasetConfig` class. It looks like all tests passed. What do you think?
2020-06-05T19:30:29Z
https://github.com/huggingface/datasets/pull/236#issuecomment-639744767
https://api.github.com/repos/huggingface/datasets/issues/236
MDEyOklzc3VlQ29tbWVudDYzOTc0NDc2Nw==
https://api.github.com/repos/huggingface/datasets/issues/comments/639744767/reactions
2020-06-05T19:30:29Z
https://api.github.com/repos/huggingface/datasets/issues/comments/639744767
https://avatars.githubusercontent.com/u/1479733?v=4
https://api.github.com/users/aleSuglia/events{/privacy}
https://api.github.com/users/aleSuglia/followers
https://api.github.com/users/aleSuglia/following{/other_user}
https://api.github.com/users/aleSuglia/gists{/gist_id}
https://github.com/aleSuglia
aleSuglia
MDQ6VXNlcjE0Nzk3MzM=
https://api.github.com/users/aleSuglia/orgs
https://api.github.com/users/aleSuglia/received_events
https://api.github.com/users/aleSuglia/repos
https://api.github.com/users/aleSuglia/starred{/owner}{/repo}
https://api.github.com/users/aleSuglia/subscriptions
User
https://api.github.com/users/aleSuglia
public
CONTRIBUTOR
great @aleSuglia. I requested an additional review from @thomwolf @lhoestq and @patrickvonplaten @jplu . You can merge it after an approval from one of them
2020-06-05T19:41:50Z
https://github.com/huggingface/datasets/pull/236#issuecomment-639752165
https://api.github.com/repos/huggingface/datasets/issues/236
MDEyOklzc3VlQ29tbWVudDYzOTc1MjE2NQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/639752165/reactions
2020-06-05T19:41:50Z
https://api.github.com/repos/huggingface/datasets/issues/comments/639752165
https://avatars.githubusercontent.com/u/38249783?v=4
https://api.github.com/users/mariamabarham/events{/privacy}
https://api.github.com/users/mariamabarham/followers
https://api.github.com/users/mariamabarham/following{/other_user}
https://api.github.com/users/mariamabarham/gists{/gist_id}
https://github.com/mariamabarham
mariamabarham
MDQ6VXNlcjM4MjQ5Nzgz
https://api.github.com/users/mariamabarham/orgs
https://api.github.com/users/mariamabarham/received_events
https://api.github.com/users/mariamabarham/repos
https://api.github.com/users/mariamabarham/starred{/owner}{/repo}
https://api.github.com/users/mariamabarham/subscriptions
User
https://api.github.com/users/mariamabarham
public
CONTRIBUTOR
Looks great! Thanks for adding the dummy data :-) 
2020-06-05T20:38:47Z
https://github.com/huggingface/datasets/pull/236#issuecomment-639802615
https://api.github.com/repos/huggingface/datasets/issues/236
MDEyOklzc3VlQ29tbWVudDYzOTgwMjYxNQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/639802615/reactions
2020-06-05T20:38:47Z
https://api.github.com/repos/huggingface/datasets/issues/comments/639802615
https://avatars.githubusercontent.com/u/23423619?v=4
https://api.github.com/users/patrickvonplaten/events{/privacy}
https://api.github.com/users/patrickvonplaten/followers
https://api.github.com/users/patrickvonplaten/following{/other_user}
https://api.github.com/users/patrickvonplaten/gists{/gist_id}
https://github.com/patrickvonplaten
patrickvonplaten
MDQ6VXNlcjIzNDIzNjE5
https://api.github.com/users/patrickvonplaten/orgs
https://api.github.com/users/patrickvonplaten/received_events
https://api.github.com/users/patrickvonplaten/repos
https://api.github.com/users/patrickvonplaten/starred{/owner}{/repo}
https://api.github.com/users/patrickvonplaten/subscriptions
User
https://api.github.com/users/patrickvonplaten
public
CONTRIBUTOR
Not sure whether it's the most appropriate place but I'll ask another design question. For Vision+Language dataset, is very common to have visual features associated with each example. At the moment, for instance, I'm only integrating the image identifier so that people can later on lookup the image features during training. Do you recommend this approach or do you think it should be done in a different way?

Thank you for your answer!
2020-06-05T20:46:44Z
https://github.com/huggingface/datasets/pull/236#issuecomment-639807442
https://api.github.com/repos/huggingface/datasets/issues/236
MDEyOklzc3VlQ29tbWVudDYzOTgwNzQ0Mg==
https://api.github.com/repos/huggingface/datasets/issues/comments/639807442/reactions
2020-06-05T20:46:44Z
https://api.github.com/repos/huggingface/datasets/issues/comments/639807442
https://avatars.githubusercontent.com/u/1479733?v=4
https://api.github.com/users/aleSuglia/events{/privacy}
https://api.github.com/users/aleSuglia/followers
https://api.github.com/users/aleSuglia/following{/other_user}
https://api.github.com/users/aleSuglia/gists{/gist_id}
https://github.com/aleSuglia
aleSuglia
MDQ6VXNlcjE0Nzk3MzM=
https://api.github.com/users/aleSuglia/orgs
https://api.github.com/users/aleSuglia/received_events
https://api.github.com/users/aleSuglia/repos
https://api.github.com/users/aleSuglia/starred{/owner}{/repo}
https://api.github.com/users/aleSuglia/subscriptions
User
https://api.github.com/users/aleSuglia
public
MEMBER
Hi @aleSuglia your remark on the visual features is a good point.

We haven't started to dive deeply into how CV datasets are usually structured (cc @sgugger)

Do you have a pointer to how visual features are currently loaded and accessed by people using GuessCompWhat? 
2020-06-05T21:07:34Z
https://github.com/huggingface/datasets/pull/236#issuecomment-639818943
https://api.github.com/repos/huggingface/datasets/issues/236
MDEyOklzc3VlQ29tbWVudDYzOTgxODk0Mw==
https://api.github.com/repos/huggingface/datasets/issues/comments/639818943/reactions
2020-06-05T21:07:34Z
https://api.github.com/repos/huggingface/datasets/issues/comments/639818943
https://avatars.githubusercontent.com/u/7353373?v=4
https://api.github.com/users/thomwolf/events{/privacy}
https://api.github.com/users/thomwolf/followers
https://api.github.com/users/thomwolf/following{/other_user}
https://api.github.com/users/thomwolf/gists{/gist_id}
https://github.com/thomwolf
thomwolf
MDQ6VXNlcjczNTMzNzM=
https://api.github.com/users/thomwolf/orgs
https://api.github.com/users/thomwolf/received_events
https://api.github.com/users/thomwolf/repos
https://api.github.com/users/thomwolf/starred{/owner}{/repo}
https://api.github.com/users/thomwolf/subscriptions
User
https://api.github.com/users/thomwolf
public
CONTRIBUTOR
@thomwolf As far as I know, people using Language+Vision tasks they typically have their reference dataset (either in JSON or JSONL format) and for each example in it they have an identifier that specifies the reference image. Currently, images are represented by either pooling-based visual features (average pooling of ResNet or VGGNet features, see [DeVries et.al, 2017](https://arxiv.org/abs/1611.08481), [Shekhar et.al, 2019](https://www.aclweb.org/anthology/N19-1265.pdf)) where you have a single vector for every image. Another option is to use a set of feature maps for every image extracted from a specific layer of a CNN (see [Xu et.al, 2015](https://arxiv.org/abs/1502.03044)). A more common and recent option, especially with large-scale multi-modal transformers [Li et. al, 2019](https://arxiv.org/abs/1908.03557), is to use FastRCNN features. 

For all these types of features, people use either HD5F or NumPy compressed representations. In my personal projects, I've ditched altogether HD5F because it doesn't have out-of-the-box support for multi-processing (unless you have an ad-hoc installation of it). I've been successfully using a NumPy compressed file for each image so that I can store any sort of information in it (see [numpy.savez](https://numpy.org/doc/stable/reference/generated/numpy.savez.html)). However, I believe that Apache Arrow would be a really good fit for this type of features. 

Looking forward to hearing your thoughts about it!
2020-06-05T21:31:55Z
https://github.com/huggingface/datasets/pull/236#issuecomment-639832029
https://api.github.com/repos/huggingface/datasets/issues/236
MDEyOklzc3VlQ29tbWVudDYzOTgzMjAyOQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/639832029/reactions
2020-06-06T07:27:45Z
https://api.github.com/repos/huggingface/datasets/issues/comments/639832029
https://avatars.githubusercontent.com/u/1479733?v=4
https://api.github.com/users/aleSuglia/events{/privacy}
https://api.github.com/users/aleSuglia/followers
https://api.github.com/users/aleSuglia/following{/other_user}
https://api.github.com/users/aleSuglia/gists{/gist_id}
https://github.com/aleSuglia
aleSuglia
MDQ6VXNlcjE0Nzk3MzM=
https://api.github.com/users/aleSuglia/orgs
https://api.github.com/users/aleSuglia/received_events
https://api.github.com/users/aleSuglia/repos
https://api.github.com/users/aleSuglia/starred{/owner}{/repo}
https://api.github.com/users/aleSuglia/subscriptions
User
https://api.github.com/users/aleSuglia
public
MEMBER
Awesome work on this one thanks :)
2020-06-11T07:45:15Z
https://github.com/huggingface/datasets/pull/236#issuecomment-642473708
https://api.github.com/repos/huggingface/datasets/issues/236
MDEyOklzc3VlQ29tbWVudDY0MjQ3MzcwOA==
https://api.github.com/repos/huggingface/datasets/issues/comments/642473708/reactions
2020-06-11T07:45:15Z
https://api.github.com/repos/huggingface/datasets/issues/comments/642473708
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
@thomwolf I was thinking that I should create an issue regarding the visual features so that we can keep track of it for future work. I think it would be great to have it in NLP and I'll be happy to contribute. Let me know what you think :) 
2020-06-11T09:43:42Z
https://github.com/huggingface/datasets/pull/236#issuecomment-642537033
https://api.github.com/repos/huggingface/datasets/issues/236
MDEyOklzc3VlQ29tbWVudDY0MjUzNzAzMw==
https://api.github.com/repos/huggingface/datasets/issues/comments/642537033/reactions
2020-06-11T09:43:42Z
https://api.github.com/repos/huggingface/datasets/issues/comments/642537033
https://avatars.githubusercontent.com/u/1479733?v=4
https://api.github.com/users/aleSuglia/events{/privacy}
https://api.github.com/users/aleSuglia/followers
https://api.github.com/users/aleSuglia/following{/other_user}
https://api.github.com/users/aleSuglia/gists{/gist_id}
https://github.com/aleSuglia
aleSuglia
MDQ6VXNlcjE0Nzk3MzM=
https://api.github.com/users/aleSuglia/orgs
https://api.github.com/users/aleSuglia/received_events
https://api.github.com/users/aleSuglia/repos
https://api.github.com/users/aleSuglia/starred{/owner}{/repo}
https://api.github.com/users/aleSuglia/subscriptions
User
https://api.github.com/users/aleSuglia
public
Hello,

Thanks for the amazing library that you put together. I'm Alessandro Suglia, the first author of CompGuessWhat?!, a recently released dataset for grounded language learning accepted to ACL 2020 ([https://compguesswhat.github.io](https://compguesswhat.github.io)).

This pull-request adds the CompGuessWhat?! splits that have been extracted from the original dataset. This is only part of our evaluation framework because there is also an additional split of the dataset that has a completely different set of games. I didn't integrate it yet because I didn't know what would be the best practice in this case. Let me clarify the scenario.

In our paper, we have a main dataset (let's call it `compguesswhat-gameplay`) and a zero-shot dataset (let's call it `compguesswhat-zs-gameplay`). In the current code of the pull-request, I have only integrated `compguesswhat-gameplay`. I was thinking that it would be nice to have the `compguesswhat-zs-gameplay` in the same dataset class by simply specifying some particular option to the `nlp.load_dataset()` factory. For instance:

```python

cgw = nlp.load_dataset("compguesswhat")
cgw_zs = nlp.load_dataset("compguesswhat", zero_shot=True)
```

The other option would be to have a separate dataset class. Any preferences?  