Add narrative qa
CONTRIBUTOR
Does it make sense to download the full stories? I remember attempting to implement this dataset a while ago and ended up with something like:
```python
    def _split_generators(self, dl_manager):
        """Returns SplitGenerators."""

        dl_dir = dl_manager.download_and_extract(_DOWNLOAD_URL)
        data_dir = os.path.join(dl_dir, "narrativeqa-master")

        urls = {"test":{}, "train": {},"valid":{}}
        with open(os.path.join(data_dir,"documents.csv")) as f_in:
            csv_reader = csv.reader(f_in)
            next(csv_reader) # discard header row
            for i,row in enumerate(csv_reader):
                if i > 1572:
                    break
                if row != []:
                    urls[row[1]][row[0]] = row[3]

        url_files = {}
        for key in urls.keys():
            url_files[key] = dl_manager.download_and_extract(urls[key])

        return [
            nlp.SplitGenerator(
                name=nlp.Split.TRAIN,
                gen_kwargs={
                    "data_dir":data_dir,
                    "split":"train",
                    "doc_id_to_path":url_files["train"]
                    }
            ),
        ....
```
It does end up cluttering your huggingface cache dir though.
2020-06-24T18:08:40Z
https://github.com/huggingface/datasets/pull/309#issuecomment-648980528
https://api.github.com/repos/huggingface/datasets/issues/309
MDEyOklzc3VlQ29tbWVudDY0ODk4MDUyOA==
https://api.github.com/repos/huggingface/datasets/issues/comments/648980528/reactions
2020-06-24T18:08:40Z
https://api.github.com/repos/huggingface/datasets/issues/comments/648980528
https://avatars.githubusercontent.com/u/13795113?v=4
https://api.github.com/users/ghomasHudson/events{/privacy}
https://api.github.com/users/ghomasHudson/followers
https://api.github.com/users/ghomasHudson/following{/other_user}
https://api.github.com/users/ghomasHudson/gists{/gist_id}
https://github.com/ghomasHudson
ghomasHudson
MDQ6VXNlcjEzNzk1MTEz
https://api.github.com/users/ghomasHudson/orgs
https://api.github.com/users/ghomasHudson/received_events
https://api.github.com/users/ghomasHudson/repos
https://api.github.com/users/ghomasHudson/starred{/owner}{/repo}
https://api.github.com/users/ghomasHudson/subscriptions
User
https://api.github.com/users/ghomasHudson
public
CONTRIBUTOR
Also since there doesn't seem to be any meaning in the order of answer_1 and answer_2, it might make sense to combine them (see [squad.py](https://github.com/huggingface/nlp/blob/8b0ffc85e4e52ae1f18d31be99b6c70b82c991ca/datasets/squad/squad.py#L86-L88)):
```python
"answers": nlp.features.Sequence({
    "text": nlp.Value("string"),
    "tokenized": nlp.features.Sequence(nlp.Value("string"))
})
```
(the tokenized features should also probably be lists of strings not just strings - see [natural_questions.py](https://github.com/huggingface/nlp/blob/4cd34287300a1135ce7b22f6dd209ca305c71b3a/datasets/natural_questions/natural_questions.py#L83))

Again, this is a personal preference thing, but it might be useful to combine the document-related features:
```python
{
    "document": {
        "id": nlp.Value("string"),
        "kind": nlp.Value("string"),
        "url": nlp.Value("string"),
        "file_size": nlp.Value("int32"),
        "word_count": nlp.Value("int32"),
        "start": nlp.Value("string"),
        "end": nlp.Value("string"),
        "wiki_url": nlp.Value("string"),
        "wiki_title": nlp.Value("string"),
        "summary": nlp.features.Sequence({
            "text": nlp.Value("string"),
            "tokens": nlp.features.Sequence(nlp.Value("string"))
        }),
        "text": nlp.Value("string"),
    },
    "question": nlp.features.Sequence({
        "text": nlp.Value("string"),
        "tokens": nlp.features.Sequence(nlp.Value("string"))
    }),
    "answers": nlp.features.Sequence({
        "text": nlp.Value("string"),
        "tokens": nlp.features.Sequence(nlp.Value("string"))
    })
}
```
2020-06-24T18:23:27Z
https://github.com/huggingface/datasets/pull/309#issuecomment-648988222
https://api.github.com/repos/huggingface/datasets/issues/309
MDEyOklzc3VlQ29tbWVudDY0ODk4ODIyMg==
https://api.github.com/repos/huggingface/datasets/issues/comments/648988222/reactions
2020-06-24T18:31:55Z
https://api.github.com/repos/huggingface/datasets/issues/comments/648988222
https://avatars.githubusercontent.com/u/13795113?v=4
https://api.github.com/users/ghomasHudson/events{/privacy}
https://api.github.com/users/ghomasHudson/followers
https://api.github.com/users/ghomasHudson/following{/other_user}
https://api.github.com/users/ghomasHudson/gists{/gist_id}
https://github.com/ghomasHudson
ghomasHudson
MDQ6VXNlcjEzNzk1MTEz
https://api.github.com/users/ghomasHudson/orgs
https://api.github.com/users/ghomasHudson/received_events
https://api.github.com/users/ghomasHudson/repos
https://api.github.com/users/ghomasHudson/starred{/owner}{/repo}
https://api.github.com/users/ghomasHudson/subscriptions
User
https://api.github.com/users/ghomasHudson
public
MEMBER
Did you manage to fix the dummy data @Varal7 ?
2020-07-15T11:53:36Z
https://github.com/huggingface/datasets/pull/309#issuecomment-658723834
https://api.github.com/repos/huggingface/datasets/issues/309
MDEyOklzc3VlQ29tbWVudDY1ODcyMzgzNA==
https://api.github.com/repos/huggingface/datasets/issues/comments/658723834/reactions
2020-07-15T11:53:42Z
https://api.github.com/repos/huggingface/datasets/issues/comments/658723834
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
@lhoestq do you think it's acceptable for the `dl_manager` to go grab all the individual stories from project gutenburg? I've got a working version of that but it does clutter up your huggingface cache somewhat.

The real value (and original purpose) of this dataset is doing question answering on the full text.
2020-07-15T11:59:19Z
https://github.com/huggingface/datasets/pull/309#issuecomment-658726039
https://api.github.com/repos/huggingface/datasets/issues/309
MDEyOklzc3VlQ29tbWVudDY1ODcyNjAzOQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/658726039/reactions
2020-07-15T11:59:19Z
https://api.github.com/repos/huggingface/datasets/issues/comments/658726039
https://avatars.githubusercontent.com/u/13795113?v=4
https://api.github.com/users/ghomasHudson/events{/privacy}
https://api.github.com/users/ghomasHudson/followers
https://api.github.com/users/ghomasHudson/following{/other_user}
https://api.github.com/users/ghomasHudson/gists{/gist_id}
https://github.com/ghomasHudson
ghomasHudson
MDQ6VXNlcjEzNzk1MTEz
https://api.github.com/users/ghomasHudson/orgs
https://api.github.com/users/ghomasHudson/received_events
https://api.github.com/users/ghomasHudson/repos
https://api.github.com/users/ghomasHudson/starred{/owner}{/repo}
https://api.github.com/users/ghomasHudson/subscriptions
User
https://api.github.com/users/ghomasHudson
public
MEMBER
> @lhoestq do you think it's acceptable for the `dl_manager` to go grab all the individual stories from project gutenburg? I've got a working version of that but it does clutter up your huggingface cache somewhat.
> 
> The real value (and original purpose) of this dataset is doing question answering on the full text.

What's the problem exactly with the cache ?
2020-07-15T12:08:27Z
https://github.com/huggingface/datasets/pull/309#issuecomment-658729903
https://api.github.com/repos/huggingface/datasets/issues/309
MDEyOklzc3VlQ29tbWVudDY1ODcyOTkwMw==
https://api.github.com/repos/huggingface/datasets/issues/comments/658729903/reactions
2020-07-15T12:08:27Z
https://api.github.com/repos/huggingface/datasets/issues/comments/658729903
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Nothing, just that because each story is a separate download it gets a bit messy as all 1573 files are under `~/.cache/hugginface/datasets` rather than organized under a subdir.

Probably doesn't matter to the end user though.
2020-07-15T12:19:15Z
https://github.com/huggingface/datasets/pull/309#issuecomment-658734251
https://api.github.com/repos/huggingface/datasets/issues/309
MDEyOklzc3VlQ29tbWVudDY1ODczNDI1MQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/658734251/reactions
2020-07-15T12:19:15Z
https://api.github.com/repos/huggingface/datasets/issues/comments/658734251
https://avatars.githubusercontent.com/u/13795113?v=4
https://api.github.com/users/ghomasHudson/events{/privacy}
https://api.github.com/users/ghomasHudson/followers
https://api.github.com/users/ghomasHudson/following{/other_user}
https://api.github.com/users/ghomasHudson/gists{/gist_id}
https://github.com/ghomasHudson
ghomasHudson
MDQ6VXNlcjEzNzk1MTEz
https://api.github.com/users/ghomasHudson/orgs
https://api.github.com/users/ghomasHudson/received_events
https://api.github.com/users/ghomasHudson/repos
https://api.github.com/users/ghomasHudson/starred{/owner}{/repo}
https://api.github.com/users/ghomasHudson/subscriptions
User
https://api.github.com/users/ghomasHudson
public
MEMBER
Yea I agree it's a mess. I just created #393 to make things easier.
2020-07-15T13:15:02Z
https://github.com/huggingface/datasets/pull/309#issuecomment-658760363
https://api.github.com/repos/huggingface/datasets/issues/309
MDEyOklzc3VlQ29tbWVudDY1ODc2MDM2Mw==
https://api.github.com/repos/huggingface/datasets/issues/comments/658760363/reactions
2020-07-15T13:15:02Z
https://api.github.com/repos/huggingface/datasets/issues/comments/658760363
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
I got the PR merged to have a cleaner the cache directory (everything is downloaded inside the 'downloads' sub-directory).
Feel free to download all the stories then @ghomasHudson @Varal7  x)
If you have the possibility of downloading a compressed file with most of the stories at once it would be better though.
2020-07-16T12:37:48Z
https://github.com/huggingface/datasets/pull/309#issuecomment-659383354
https://api.github.com/repos/huggingface/datasets/issues/309
MDEyOklzc3VlQ29tbWVudDY1OTM4MzM1NA==
https://api.github.com/repos/huggingface/datasets/issues/comments/659383354/reactions
2020-07-16T12:37:48Z
https://api.github.com/repos/huggingface/datasets/issues/comments/659383354
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Looks good @lhoestq . The problem I'm having at the moment is that stories from project Gutenberg occasionally fail. All books are out of copyright so we should be able to host them. 

Here's a zip file of the full text if we have anywhere to put them: https://drive.google.com/file/d/17jOR7NqvzDwSlPXrlHaYV-PGI8JG-KY5/view?usp=sharing

2020-07-17T08:50:15Z
https://github.com/huggingface/datasets/pull/309#issuecomment-659970415
https://api.github.com/repos/huggingface/datasets/issues/309
MDEyOklzc3VlQ29tbWVudDY1OTk3MDQxNQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/659970415/reactions
2020-07-17T08:50:45Z
https://api.github.com/repos/huggingface/datasets/issues/comments/659970415
https://avatars.githubusercontent.com/u/13795113?v=4
https://api.github.com/users/ghomasHudson/events{/privacy}
https://api.github.com/users/ghomasHudson/followers
https://api.github.com/users/ghomasHudson/following{/other_user}
https://api.github.com/users/ghomasHudson/gists{/gist_id}
https://github.com/ghomasHudson
ghomasHudson
MDQ6VXNlcjEzNzk1MTEz
https://api.github.com/users/ghomasHudson/orgs
https://api.github.com/users/ghomasHudson/received_events
https://api.github.com/users/ghomasHudson/repos
https://api.github.com/users/ghomasHudson/starred{/owner}{/repo}
https://api.github.com/users/ghomasHudson/subscriptions
User
https://api.github.com/users/ghomasHudson
public
MEMBER
I put the zip file here @ghomasHudson 
https://storage.googleapis.com/huggingface-nlp/datasets/narrative_qa/narrativeqa_full_text.zip

Sorry for the delay
2020-08-12T08:49:37Z
https://github.com/huggingface/datasets/pull/309#issuecomment-672742756
https://api.github.com/repos/huggingface/datasets/issues/309
MDEyOklzc3VlQ29tbWVudDY3Mjc0Mjc1Ng==
https://api.github.com/repos/huggingface/datasets/issues/comments/672742756/reactions
2020-08-12T08:49:37Z
https://api.github.com/repos/huggingface/datasets/issues/comments/672742756
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
Closing in favor of #499
2020-09-03T09:02:09Z
https://github.com/huggingface/datasets/pull/309#issuecomment-686355934
https://api.github.com/repos/huggingface/datasets/issues/309
MDEyOklzc3VlQ29tbWVudDY4NjM1NTkzNA==
https://api.github.com/repos/huggingface/datasets/issues/comments/686355934/reactions
2020-09-03T09:02:09Z
https://api.github.com/repos/huggingface/datasets/issues/comments/686355934
https://avatars.githubusercontent.com/u/7353373?v=4
https://api.github.com/users/thomwolf/events{/privacy}
https://api.github.com/users/thomwolf/followers
https://api.github.com/users/thomwolf/following{/other_user}
https://api.github.com/users/thomwolf/gists{/gist_id}
https://github.com/thomwolf
thomwolf
MDQ6VXNlcjczNTMzNzM=
https://api.github.com/users/thomwolf/orgs
https://api.github.com/users/thomwolf/received_events
https://api.github.com/users/thomwolf/repos
https://api.github.com/users/thomwolf/starred{/owner}{/repo}
https://api.github.com/users/thomwolf/subscriptions
User
https://api.github.com/users/thomwolf
public
Test cases for dummy data don't pass

Only contains data for summaries (not whole story)