add inquisitive
CONTRIBUTOR
`dummy_data` right now contains all article files, keeping only the required articles for dummy data fails the dummy data test.
Any idea ?
2020-11-30T17:49:43Z
https://github.com/huggingface/datasets/pull/926#issuecomment-735940528
https://api.github.com/repos/huggingface/datasets/issues/926
MDEyOklzc3VlQ29tbWVudDczNTk0MDUyOA==
https://api.github.com/repos/huggingface/datasets/issues/comments/735940528/reactions
2020-11-30T17:49:43Z
https://api.github.com/repos/huggingface/datasets/issues/comments/735940528
https://avatars.githubusercontent.com/u/27137566?v=4
https://api.github.com/users/patil-suraj/events{/privacy}
https://api.github.com/users/patil-suraj/followers
https://api.github.com/users/patil-suraj/following{/other_user}
https://api.github.com/users/patil-suraj/gists{/gist_id}
https://github.com/patil-suraj
patil-suraj
MDQ6VXNlcjI3MTM3NTY2
https://api.github.com/users/patil-suraj/orgs
https://api.github.com/users/patil-suraj/received_events
https://api.github.com/users/patil-suraj/repos
https://api.github.com/users/patil-suraj/starred{/owner}{/repo}
https://api.github.com/users/patil-suraj/subscriptions
User
https://api.github.com/users/patil-suraj
public
MEMBER
> `dummy_data` right now contains all article files, keeping only the required articles for dummy data fails the dummy data test.
> Any idea ?

We should definitely find a way to make it work with only a few articles.

If it doesn't work right now for dummy data, I guess it's because it tries to load every single article file ?

If so, then maybe you can use `os.listdir` method to first check all the data files available in the path where the `articles.tgz` file is extracted. Then you can simply iter through the data files and depending on their ID, include them in the train or test set. With this method you should be able to have only a few articles files per split in the dummy data. Does that make sense ?
2020-11-30T18:14:37Z
https://github.com/huggingface/datasets/pull/926#issuecomment-735954064
https://api.github.com/repos/huggingface/datasets/issues/926
MDEyOklzc3VlQ29tbWVudDczNTk1NDA2NA==
https://api.github.com/repos/huggingface/datasets/issues/comments/735954064/reactions
2020-11-30T18:14:37Z
https://api.github.com/repos/huggingface/datasets/issues/comments/735954064
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
fixed! so the issue was, `articles_ids` were prepared based on the number of files in articles dir, so for dummy data questions it was not able to load some articles due to incorrect ids and the test was failing
2020-11-30T18:36:23Z
https://github.com/huggingface/datasets/pull/926#issuecomment-735965727
https://api.github.com/repos/huggingface/datasets/issues/926
MDEyOklzc3VlQ29tbWVudDczNTk2NTcyNw==
https://api.github.com/repos/huggingface/datasets/issues/comments/735965727/reactions
2020-11-30T18:36:52Z
https://api.github.com/repos/huggingface/datasets/issues/comments/735965727
https://avatars.githubusercontent.com/u/27137566?v=4
https://api.github.com/users/patil-suraj/events{/privacy}
https://api.github.com/users/patil-suraj/followers
https://api.github.com/users/patil-suraj/following{/other_user}
https://api.github.com/users/patil-suraj/gists{/gist_id}
https://github.com/patil-suraj
patil-suraj
MDQ6VXNlcjI3MTM3NTY2
https://api.github.com/users/patil-suraj/orgs
https://api.github.com/users/patil-suraj/received_events
https://api.github.com/users/patil-suraj/repos
https://api.github.com/users/patil-suraj/starred{/owner}{/repo}
https://api.github.com/users/patil-suraj/subscriptions
User
https://api.github.com/users/patil-suraj
public
Adding inquisitive qg dataset

More info: https://github.com/wjko2/INQUISITIVE