add openwebtext2
CONTRIBUTOR
It seems we need to `pip install jsonlines` to pass the checks ?
2021-08-14T08:00:55Z
https://github.com/huggingface/datasets/pull/2802#issuecomment-898862160
https://api.github.com/repos/huggingface/datasets/issues/2802
IC_kwDODunzps41k4xQ
https://api.github.com/repos/huggingface/datasets/issues/comments/898862160/reactions
2021-08-14T08:00:55Z
https://api.github.com/repos/huggingface/datasets/issues/comments/898862160
https://avatars.githubusercontent.com/u/17963619?v=4
https://api.github.com/users/richarddwang/events{/privacy}
https://api.github.com/users/richarddwang/followers
https://api.github.com/users/richarddwang/following{/other_user}
https://api.github.com/users/richarddwang/gists{/gist_id}
https://github.com/richarddwang
richarddwang
MDQ6VXNlcjE3OTYzNjE5
https://api.github.com/users/richarddwang/orgs
https://api.github.com/users/richarddwang/received_events
https://api.github.com/users/richarddwang/repos
https://api.github.com/users/richarddwang/starred{/owner}{/repo}
https://api.github.com/users/richarddwang/subscriptions
User
https://api.github.com/users/richarddwang
public
MEMBER
Hi ! Do you really need `jsonlines` ? I think it simply uses `json.loads` under the hood.

Currently the test are failing because `jsonlines` is not part of the extra requirements `TESTS_REQUIRE` in setup.py

So either you can replace `jsonlines` with a simple for loop on the lines of the files and use `json.loads`, or you can add `TESTS_REQUIRE` to the test requirements (but in this case users will have to install it as well).
2021-08-18T15:09:13Z
https://github.com/huggingface/datasets/pull/2802#issuecomment-901195417
https://api.github.com/repos/huggingface/datasets/issues/2802
IC_kwDODunzps41tyaZ
https://api.github.com/repos/huggingface/datasets/issues/comments/901195417/reactions
2021-08-18T15:09:13Z
https://api.github.com/repos/huggingface/datasets/issues/comments/901195417
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Thanks for your suggestion. I now know `io` and json lines format better and has changed `jsonlines` to just `readlines`.
2021-08-19T13:36:54Z
https://github.com/huggingface/datasets/pull/2802#issuecomment-901922389
https://api.github.com/repos/huggingface/datasets/issues/2802
IC_kwDODunzps41wj5V
https://api.github.com/repos/huggingface/datasets/issues/comments/901922389/reactions
2021-08-20T00:22:19Z
https://api.github.com/repos/huggingface/datasets/issues/comments/901922389
https://avatars.githubusercontent.com/u/17963619?v=4
https://api.github.com/users/richarddwang/events{/privacy}
https://api.github.com/users/richarddwang/followers
https://api.github.com/users/richarddwang/following{/other_user}
https://api.github.com/users/richarddwang/gists{/gist_id}
https://github.com/richarddwang
richarddwang
MDQ6VXNlcjE3OTYzNjE5
https://api.github.com/users/richarddwang/orgs
https://api.github.com/users/richarddwang/received_events
https://api.github.com/users/richarddwang/repos
https://api.github.com/users/richarddwang/starred{/owner}{/repo}
https://api.github.com/users/richarddwang/subscriptions
User
https://api.github.com/users/richarddwang
public
openwebtext2 is part of EleutherAI/The Pile, but AFAIK, The Pile dataset blend all sub datasets together thus we are not able to use just one of its sub dataset from The Pile data. So I create an independent dataset using The Pile preliminary components.

When I was creating dataset card. I found there is room for creating / editing dataset card. I've made it an issue. #2797

Also I am wondering whether the import of The Pile dataset is actively undertaken (because I may need it recently)? #1675