Use standard open-domain validation split in nq_open
CONTRIBUTOR
I had to run datasets-cli with --ignore_verifications the first time since it was complaining about a missing file, but now it runs without that flag fine. I moved dummy_data.zip to the new folder, but also had to modify the filename of the test file in the zip (should I not have done that?). Finally, I added the pretty name tag.
2021-10-05T14:41:45Z
https://github.com/huggingface/datasets/pull/3029#issuecomment-934475444
https://api.github.com/repos/huggingface/datasets/issues/3029
IC_kwDODunzps43sva0
https://api.github.com/repos/huggingface/datasets/issues/comments/934475444/reactions
2021-10-05T14:41:45Z
https://api.github.com/repos/huggingface/datasets/issues/comments/934475444
https://avatars.githubusercontent.com/u/417568?v=4
https://api.github.com/users/craffel/events{/privacy}
https://api.github.com/users/craffel/followers
https://api.github.com/users/craffel/following{/other_user}
https://api.github.com/users/craffel/gists{/gist_id}
https://github.com/craffel
craffel
MDQ6VXNlcjQxNzU2OA==
https://api.github.com/users/craffel/orgs
https://api.github.com/users/craffel/received_events
https://api.github.com/users/craffel/repos
https://api.github.com/users/craffel/starred{/owner}{/repo}
https://api.github.com/users/craffel/subscriptions
User
https://api.github.com/users/craffel
public
CONTRIBUTOR
Great, thanks for the help.
2021-10-05T14:50:26Z
https://github.com/huggingface/datasets/pull/3029#issuecomment-934482584
https://api.github.com/repos/huggingface/datasets/issues/3029
IC_kwDODunzps43sxKY
https://api.github.com/repos/huggingface/datasets/issues/comments/934482584/reactions
2021-10-05T14:50:26Z
https://api.github.com/repos/huggingface/datasets/issues/comments/934482584
https://avatars.githubusercontent.com/u/417568?v=4
https://api.github.com/users/craffel/events{/privacy}
https://api.github.com/users/craffel/followers
https://api.github.com/users/craffel/following{/other_user}
https://api.github.com/users/craffel/gists{/gist_id}
https://github.com/craffel
craffel
MDQ6VXNlcjQxNzU2OA==
https://api.github.com/users/craffel/orgs
https://api.github.com/users/craffel/received_events
https://api.github.com/users/craffel/repos
https://api.github.com/users/craffel/starred{/owner}{/repo}
https://api.github.com/users/craffel/subscriptions
User
https://api.github.com/users/craffel
public
The nq_open dataset originally drew the validation set from this file:
https://github.com/google-research-datasets/natural-questions/blob/master/nq_open/NQ-open.efficientqa.dev.1.1.sample.jsonl
However, that's the dev set used specifically and only for the efficientqa competition, and it's not the same dev set as is used in every open-domain question answering paper (including the Lee et al paper that introduced the open-domain variant of NQ, cited at the top of the dataset file). This PR changes nq_open to use the standard validation split and bumps the version to 2.0.0 since this is a breaking change.