add Toronto BooksCorpus
MEMBER
Thanks for adding this one !

About the three points you mentioned:
1. I think the `toronto_books_corpus` branch can be removed @mariamabarham ? 
2. You can use the download manager to download from google drive. For you case you can just do something like 
```python
URL = "https://drive.google.com/uc?export=download&id=16KCjV9z_FHm8LgZw05RSuk4EsAWPOP_z"
...
arch_path = dl_manager.download_and_extract(URL)
```
Also this is is an unofficial host of the dataset, we should probably host it ourselves if we can.
3. Not sure about the copyright here, but I maybe @thomwolf has better insights about it. 
2020-06-08T12:40:25Z
https://github.com/huggingface/datasets/pull/248#issuecomment-640577819
https://api.github.com/repos/huggingface/datasets/issues/248
MDEyOklzc3VlQ29tbWVudDY0MDU3NzgxOQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/640577819/reactions
2020-06-08T12:40:25Z
https://api.github.com/repos/huggingface/datasets/issues/comments/640577819
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Yes it can be removed
2020-06-08T14:05:54Z
https://github.com/huggingface/datasets/pull/248#issuecomment-640630636
https://api.github.com/repos/huggingface/datasets/issues/248
MDEyOklzc3VlQ29tbWVudDY0MDYzMDYzNg==
https://api.github.com/repos/huggingface/datasets/issues/comments/640630636/reactions
2020-06-08T14:05:54Z
https://api.github.com/repos/huggingface/datasets/issues/comments/640630636
https://avatars.githubusercontent.com/u/38249783?v=4
https://api.github.com/users/mariamabarham/events{/privacy}
https://api.github.com/users/mariamabarham/followers
https://api.github.com/users/mariamabarham/following{/other_user}
https://api.github.com/users/mariamabarham/gists{/gist_id}
https://github.com/mariamabarham
mariamabarham
MDQ6VXNlcjM4MjQ5Nzgz
https://api.github.com/users/mariamabarham/orgs
https://api.github.com/users/mariamabarham/received_events
https://api.github.com/users/mariamabarham/repos
https://api.github.com/users/mariamabarham/starred{/owner}{/repo}
https://api.github.com/users/mariamabarham/subscriptions
User
https://api.github.com/users/mariamabarham
public
MEMBER
I just downloaded the file and put it on gs. The public url is
https://storage.googleapis.com/huggingface-nlp/datasets/toronto_books_corpus/bookcorpus.tar.bz2

Could you try to change the url to this one and heck that everything is ok ?
2020-06-11T08:22:21Z
https://github.com/huggingface/datasets/pull/248#issuecomment-642492387
https://api.github.com/repos/huggingface/datasets/issues/248
MDEyOklzc3VlQ29tbWVudDY0MjQ5MjM4Nw==
https://api.github.com/repos/huggingface/datasets/issues/comments/642492387/reactions
2020-06-11T08:22:21Z
https://api.github.com/repos/huggingface/datasets/issues/comments/642492387
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
In `books.py`
```
URL = "https://storage.googleapis.com/huggingface-nlp/datasets/toronto_books_corpus/bookcorpus.tar.bz2"
```
```
Python 3.7.6 (default, Jan  8 2020, 19:59:22) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from nlp import load_dataset
>>> book = load_dataset("nlp/datasets/bookscorpus/books.py", cache_dir='~/tmp')
Downloading and preparing dataset bookscorpus/plain_text (download: 1.10 GiB, generated: 4.52 GiB, total: 5.62 GiB) to /home/yisiang/tmp/bookscorpus/plain_text/1.0.0...
Downloading: 100%|███████████████████████████████████████████████████████████| 1.18G/1.18G [00:39<00:00, 30.0MB/s]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/yisiang/nlp/src/nlp/load.py", line 520, in load_dataset
    save_infos=save_infos,
  File "/home/yisiang/nlp/src/nlp/builder.py", line 420, in download_and_prepare
    dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
  File "/home/yisiang/nlp/src/nlp/builder.py", line 460, in _download_and_prepare
    verify_checksums(self.info.download_checksums, dl_manager.get_recorded_sizes_checksums())
  File "/home/yisiang/nlp/src/nlp/utils/info_utils.py", line 31, in verify_checksums
    raise ExpectedMoreDownloadedFiles(str(set(expected_checksums) - set(recorded_checksums)))
nlp.utils.info_utils.ExpectedMoreDownloadedFiles: {'16KCjV9z_FHm8LgZw05RSuk4EsAWPOP_z'}
>>>
```

BTW, I notice the path `huggingface-nlp/datasets/toronto_books_corpus`, does it mean I have to change  folder name "bookscorpus" to "toronto_books_corpus"
2020-06-11T09:53:19Z
https://github.com/huggingface/datasets/pull/248#issuecomment-642541661
https://api.github.com/repos/huggingface/datasets/issues/248
MDEyOklzc3VlQ29tbWVudDY0MjU0MTY2MQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/642541661/reactions
2020-06-11T10:02:45Z
https://api.github.com/repos/huggingface/datasets/issues/comments/642541661
https://avatars.githubusercontent.com/u/17963619?v=4
https://api.github.com/users/richarddwang/events{/privacy}
https://api.github.com/users/richarddwang/followers
https://api.github.com/users/richarddwang/following{/other_user}
https://api.github.com/users/richarddwang/gists{/gist_id}
https://github.com/richarddwang
richarddwang
MDQ6VXNlcjE3OTYzNjE5
https://api.github.com/users/richarddwang/orgs
https://api.github.com/users/richarddwang/received_events
https://api.github.com/users/richarddwang/repos
https://api.github.com/users/richarddwang/starred{/owner}{/repo}
https://api.github.com/users/richarddwang/subscriptions
User
https://api.github.com/users/richarddwang
public
MEMBER
> In `books.py`
> 
> ```
> URL = "https://storage.googleapis.com/huggingface-nlp/datasets/toronto_books_corpus/bookcorpus.tar.bz2"
> ```
> 
> ```
> Python 3.7.6 (default, Jan  8 2020, 19:59:22) 
> [GCC 7.3.0] :: Anaconda, Inc. on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> from nlp import load_dataset
> >>> book = load_dataset("nlp/datasets/bookscorpus/books.py", cache_dir='~/tmp')
> Downloading and preparing dataset bookscorpus/plain_text (download: 1.10 GiB, generated: 4.52 GiB, total: 5.62 GiB) to /home/yisiang/tmp/bookscorpus/plain_text/1.0.0...
> Downloading: 100%|███████████████████████████████████████████████████████████| 1.18G/1.18G [00:39<00:00, 30.0MB/s]
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "/home/yisiang/nlp/src/nlp/load.py", line 520, in load_dataset
>     save_infos=save_infos,
>   File "/home/yisiang/nlp/src/nlp/builder.py", line 420, in download_and_prepare
>     dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
>   File "/home/yisiang/nlp/src/nlp/builder.py", line 460, in _download_and_prepare
>     verify_checksums(self.info.download_checksums, dl_manager.get_recorded_sizes_checksums())
>   File "/home/yisiang/nlp/src/nlp/utils/info_utils.py", line 31, in verify_checksums
>     raise ExpectedMoreDownloadedFiles(str(set(expected_checksums) - set(recorded_checksums)))
> nlp.utils.info_utils.ExpectedMoreDownloadedFiles: {'16KCjV9z_FHm8LgZw05RSuk4EsAWPOP_z'}
> >>>
> ```
> 
> BTW, I notice the path `huggingface-nlp/datasets/toronto_books_corpus`, does it mean I have to change folder name "bookscorpus" to "toronto_books_corpus"

Let me change the url to match "bookscorpus", so that you don't have to change anything. Good catch.

About the error you're getting: you just have to remove the `dataset_infos.json` and regenerate it
2020-06-11T10:08:27Z
https://github.com/huggingface/datasets/pull/248#issuecomment-642549008
https://api.github.com/repos/huggingface/datasets/issues/248
MDEyOklzc3VlQ29tbWVudDY0MjU0OTAwOA==
https://api.github.com/repos/huggingface/datasets/issues/comments/642549008/reactions
2020-06-11T10:09:50Z
https://api.github.com/repos/huggingface/datasets/issues/comments/642549008
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
The new url is https://storage.googleapis.com/huggingface-nlp/datasets/bookscorpus/bookcorpus.tar.bz2
2020-06-11T10:15:03Z
https://github.com/huggingface/datasets/pull/248#issuecomment-642551967
https://api.github.com/repos/huggingface/datasets/issues/248
MDEyOklzc3VlQ29tbWVudDY0MjU1MTk2Nw==
https://api.github.com/repos/huggingface/datasets/issues/comments/642551967/reactions
2020-06-11T10:15:03Z
https://api.github.com/repos/huggingface/datasets/issues/comments/642551967
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Hi, I found I made a mistake. I found the ELECTRA paper refer it as "BooksCorpus", but actually it is caleld "BookCorpus", according to the original paper. Sorry, I should have checked the original paper .

Can you do me a favor and change the url path to ` https://storage.googleapis.com/huggingface-nlp/datasets/bookcorpus/bookcorpus.tar.bz2` ?
2020-06-11T11:53:43Z
https://github.com/huggingface/datasets/pull/248#issuecomment-642594614
https://api.github.com/repos/huggingface/datasets/issues/248
MDEyOklzc3VlQ29tbWVudDY0MjU5NDYxNA==
https://api.github.com/repos/huggingface/datasets/issues/comments/642594614/reactions
2020-06-11T11:53:43Z
https://api.github.com/repos/huggingface/datasets/issues/comments/642594614
https://avatars.githubusercontent.com/u/17963619?v=4
https://api.github.com/users/richarddwang/events{/privacy}
https://api.github.com/users/richarddwang/followers
https://api.github.com/users/richarddwang/following{/other_user}
https://api.github.com/users/richarddwang/gists{/gist_id}
https://github.com/richarddwang
richarddwang
MDQ6VXNlcjE3OTYzNjE5
https://api.github.com/users/richarddwang/orgs
https://api.github.com/users/richarddwang/received_events
https://api.github.com/users/richarddwang/repos
https://api.github.com/users/richarddwang/starred{/owner}{/repo}
https://api.github.com/users/richarddwang/subscriptions
User
https://api.github.com/users/richarddwang
public
MEMBER
Yep I'm doing it right now. Could you please rename all the references to `bookscorpus` and `BooksCorpus` to `book_corpus` and `BookCorpus` (with the right casing) ?
2020-06-11T14:38:07Z
https://github.com/huggingface/datasets/pull/248#issuecomment-642700616
https://api.github.com/repos/huggingface/datasets/issues/248
MDEyOklzc3VlQ29tbWVudDY0MjcwMDYxNg==
https://api.github.com/repos/huggingface/datasets/issues/comments/642700616/reactions
2020-06-11T15:00:14Z
https://api.github.com/repos/huggingface/datasets/issues/comments/642700616
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Thank you @lhoestq ,
Just to confirm it fits your naming convention
* make the file path `book_corpus/book_corpus.py` ?
* make `class Bookscorpus(nlp.GeneratorBasedBuilder)` -> `BookCorpus`  (which make cache folder name `book_corpus` and user use `load_dataset('book_corpus')`) ?
(Cuz I found "HellaSwag" dataset is named "nlp/datasets/hellaswag" and `class Hellaswag` )
2020-06-11T15:03:06Z
https://github.com/huggingface/datasets/pull/248#issuecomment-642718883
https://api.github.com/repos/huggingface/datasets/issues/248
MDEyOklzc3VlQ29tbWVudDY0MjcxODg4Mw==
https://api.github.com/repos/huggingface/datasets/issues/comments/642718883/reactions
2020-06-11T15:53:38Z
https://api.github.com/repos/huggingface/datasets/issues/comments/642718883
https://avatars.githubusercontent.com/u/17963619?v=4
https://api.github.com/users/richarddwang/events{/privacy}
https://api.github.com/users/richarddwang/followers
https://api.github.com/users/richarddwang/following{/other_user}
https://api.github.com/users/richarddwang/gists{/gist_id}
https://github.com/richarddwang
richarddwang
MDQ6VXNlcjE3OTYzNjE5
https://api.github.com/users/richarddwang/orgs
https://api.github.com/users/richarddwang/received_events
https://api.github.com/users/richarddwang/repos
https://api.github.com/users/richarddwang/starred{/owner}{/repo}
https://api.github.com/users/richarddwang/subscriptions
User
https://api.github.com/users/richarddwang
public
MEMBER
Oh yea you're right about the Hellaswag example. We should keep the "_" symbol to replace spaces. As there are no space in BookCorpus, what we should do here is use:
- class name: 'Bookcorpus'
- script name: `bookcorpus/bookcorpus.py`
- use url https://storage.googleapis.com/huggingface-nlp/datasets/bookcorpus/bookcorpus.tar.bz2
And therefore the dataset name will be `bookcorpus`

Don't forget to regenerate the `dataset_infos.json` and we'll be good :D 
2020-06-11T15:37:06Z
https://github.com/huggingface/datasets/pull/248#issuecomment-642743276
https://api.github.com/repos/huggingface/datasets/issues/248
MDEyOklzc3VlQ29tbWVudDY0Mjc0MzI3Ng==
https://api.github.com/repos/huggingface/datasets/issues/comments/642743276/reactions
2020-06-11T15:37:06Z
https://api.github.com/repos/huggingface/datasets/issues/comments/642743276
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
Awesome thanks :)
2020-06-12T08:00:18Z
https://github.com/huggingface/datasets/pull/248#issuecomment-643132761
https://api.github.com/repos/huggingface/datasets/issues/248
MDEyOklzc3VlQ29tbWVudDY0MzEzMjc2MQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/643132761/reactions
2020-06-12T08:00:18Z
https://api.github.com/repos/huggingface/datasets/issues/comments/643132761
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
1. I knew there is a branch `toronto_books_corpus`
 - After I downloaded it, I found it is all non-english, and only have one row. 
- It seems that it cites the wrong paper
- according to papar using it, it is called `BooksCorpus` but not `TornotoBooksCorpus`

2. It use a text mirror in google drive
- `bookscorpus.py` include a function `download_file_from_google_drive` , maybe you will want to put it elsewhere.
- text mirror is found in this [comment on the issue](https://github.com/soskek/bookcorpus/issues/24#issuecomment-556024973), and it said to have the same statistics as the one in the paper.
- You may want to download it and put it on your gs in case of it disappears someday.

3. Copyright ?
The paper has said

> **The BookCorpus Dataset.** In order to train our sentence similarity model we collected a corpus of 11,038 books ***from the web***. These are __**free books written by yet unpublished authors**__. We only included books that had more than 20K words in order to filter out perhaps noisier shorter stories. The dataset has books in 16 different genres, e.g., Romance (2,865 books), Fantasy (1,479), Science fiction (786), Teen (430), etc. Table 2 highlights the summary statistics of our book corpus.

and we have changed the form (not books), so I don't think it should have that problems. Or we can state that use it at your own risk or only for academic use. I know @thomwolf should know these things more.

This should solved #131 