Enable streaming dataset to use the "all" split
NONE
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_4698). All of your documentation changes will be reflected on that endpoint.
2022-07-18T07:54:22Z
https://github.com/huggingface/datasets/pull/4698#issuecomment-1186882355
https://api.github.com/repos/huggingface/datasets/issues/4698
IC_kwDODunzps5GvmMz
https://api.github.com/repos/huggingface/datasets/issues/comments/1186882355/reactions
2022-07-18T07:54:22Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1186882355
https://avatars.githubusercontent.com/u/99929124?v=4
https://api.github.com/users/HuggingFaceDocBuilderDev/events{/privacy}
https://api.github.com/users/HuggingFaceDocBuilderDev/followers
https://api.github.com/users/HuggingFaceDocBuilderDev/following{/other_user}
https://api.github.com/users/HuggingFaceDocBuilderDev/gists{/gist_id}
https://github.com/HuggingFaceDocBuilderDev
HuggingFaceDocBuilderDev
U_kgDOBfTMJA
https://api.github.com/users/HuggingFaceDocBuilderDev/orgs
https://api.github.com/users/HuggingFaceDocBuilderDev/received_events
https://api.github.com/users/HuggingFaceDocBuilderDev/repos
https://api.github.com/users/HuggingFaceDocBuilderDev/starred{/owner}{/repo}
https://api.github.com/users/HuggingFaceDocBuilderDev/subscriptions
User
https://api.github.com/users/HuggingFaceDocBuilderDev
public
CONTRIBUTOR
@albertvillanova 
Adding the validation split causes these two `assert_called_once` assertions to fail with `AssertionError: Expected 'ArrowWriter' to have been called once. Called 2 times`:

https://github.com/huggingface/datasets/blob/main/tests/test_builder.py#L548-L562

It might be better to create a new dummy generator for the streaming tests, WDYT? Alternatively we could test for `self.call_count` equalling 2.
2022-07-18T07:54:48Z
https://github.com/huggingface/datasets/pull/4698#issuecomment-1186882712
https://api.github.com/repos/huggingface/datasets/issues/4698
IC_kwDODunzps5GvmSY
https://api.github.com/repos/huggingface/datasets/issues/comments/1186882712/reactions
2022-07-18T08:17:40Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1186882712
https://avatars.githubusercontent.com/u/3664563?v=4
https://api.github.com/users/cakiki/events{/privacy}
https://api.github.com/users/cakiki/followers
https://api.github.com/users/cakiki/following{/other_user}
https://api.github.com/users/cakiki/gists{/gist_id}
https://github.com/cakiki
cakiki
MDQ6VXNlcjM2NjQ1NjM=
https://api.github.com/users/cakiki/orgs
https://api.github.com/users/cakiki/received_events
https://api.github.com/users/cakiki/repos
https://api.github.com/users/cakiki/starred{/owner}{/repo}
https://api.github.com/users/cakiki/subscriptions
User
https://api.github.com/users/cakiki
public
MEMBER
@cakiki have you read my comment in the issue page?
https://github.com/huggingface/datasets/issues/4637#issuecomment-1175984812
2022-07-21T13:06:23Z
https://github.com/huggingface/datasets/pull/4698#issuecomment-1191462432
https://api.github.com/repos/huggingface/datasets/issues/4698
IC_kwDODunzps5HBEYg
https://api.github.com/repos/huggingface/datasets/issues/comments/1191462432/reactions
2022-07-21T13:06:23Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1191462432
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
CONTRIBUTOR
Streaming with `split=all` seems to be working, will fix the failing test next
2022-08-01T21:05:00Z
https://github.com/huggingface/datasets/pull/4698#issuecomment-1201719187
https://api.github.com/repos/huggingface/datasets/issues/4698
IC_kwDODunzps5HoMeT
https://api.github.com/repos/huggingface/datasets/issues/comments/1201719187/reactions
2022-08-01T21:05:00Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1201719187
https://avatars.githubusercontent.com/u/3664563?v=4
https://api.github.com/users/cakiki/events{/privacy}
https://api.github.com/users/cakiki/followers
https://api.github.com/users/cakiki/following{/other_user}
https://api.github.com/users/cakiki/gists{/gist_id}
https://github.com/cakiki
cakiki
MDQ6VXNlcjM2NjQ1NjM=
https://api.github.com/users/cakiki/orgs
https://api.github.com/users/cakiki/received_events
https://api.github.com/users/cakiki/repos
https://api.github.com/users/cakiki/starred{/owner}{/repo}
https://api.github.com/users/cakiki/subscriptions
User
https://api.github.com/users/cakiki
public
CONTRIBUTOR
Not sure if marking the PR as "ready for review" actually notified you, so tagging @albertvillanova just in case :smiley_cat: 
2022-08-10T08:35:33Z
https://github.com/huggingface/datasets/pull/4698#issuecomment-1210345373
https://api.github.com/repos/huggingface/datasets/issues/4698
IC_kwDODunzps5IJGed
https://api.github.com/repos/huggingface/datasets/issues/comments/1210345373/reactions
2022-08-10T08:35:33Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1210345373
https://avatars.githubusercontent.com/u/3664563?v=4
https://api.github.com/users/cakiki/events{/privacy}
https://api.github.com/users/cakiki/followers
https://api.github.com/users/cakiki/following{/other_user}
https://api.github.com/users/cakiki/gists{/gist_id}
https://github.com/cakiki
cakiki
MDQ6VXNlcjM2NjQ1NjM=
https://api.github.com/users/cakiki/orgs
https://api.github.com/users/cakiki/received_events
https://api.github.com/users/cakiki/repos
https://api.github.com/users/cakiki/starred{/owner}{/repo}
https://api.github.com/users/cakiki/subscriptions
User
https://api.github.com/users/cakiki
public
CONTRIBUTOR
cc @lhoestq 
2022-09-11T18:46:37Z
https://github.com/huggingface/datasets/pull/4698#issuecomment-1243020706
https://api.github.com/repos/huggingface/datasets/issues/4698
IC_kwDODunzps5KFv2i
https://api.github.com/repos/huggingface/datasets/issues/comments/1243020706/reactions
2022-09-11T18:46:37Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1243020706
https://avatars.githubusercontent.com/u/3664563?v=4
https://api.github.com/users/cakiki/events{/privacy}
https://api.github.com/users/cakiki/followers
https://api.github.com/users/cakiki/following{/other_user}
https://api.github.com/users/cakiki/gists{/gist_id}
https://github.com/cakiki
cakiki
MDQ6VXNlcjM2NjQ1NjM=
https://api.github.com/users/cakiki/orgs
https://api.github.com/users/cakiki/received_events
https://api.github.com/users/cakiki/repos
https://api.github.com/users/cakiki/starred{/owner}{/repo}
https://api.github.com/users/cakiki/subscriptions
User
https://api.github.com/users/cakiki
public
MEMBER
Hi @cakiki, still interested in working on this? :) 
2023-01-18T16:19:37Z
https://github.com/huggingface/datasets/pull/4698#issuecomment-1387337793
https://api.github.com/repos/huggingface/datasets/issues/4698
IC_kwDODunzps5SsRhB
https://api.github.com/repos/huggingface/datasets/issues/comments/1387337793/reactions
2023-01-18T16:19:37Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1387337793
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
CONTRIBUTOR
@albertvillanova So sorry; I have no idea how this slipped through the cracks. Yes, I'd still like to work on this. Is it okay if I DM you on slack?
2023-01-19T10:02:50Z
https://github.com/huggingface/datasets/pull/4698#issuecomment-1396716669
https://api.github.com/repos/huggingface/datasets/issues/4698
IC_kwDODunzps5TQDR9
https://api.github.com/repos/huggingface/datasets/issues/comments/1396716669/reactions
2023-01-19T10:02:50Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1396716669
https://avatars.githubusercontent.com/u/3664563?v=4
https://api.github.com/users/cakiki/events{/privacy}
https://api.github.com/users/cakiki/followers
https://api.github.com/users/cakiki/following{/other_user}
https://api.github.com/users/cakiki/gists{/gist_id}
https://github.com/cakiki
cakiki
MDQ6VXNlcjM2NjQ1NjM=
https://api.github.com/users/cakiki/orgs
https://api.github.com/users/cakiki/received_events
https://api.github.com/users/cakiki/repos
https://api.github.com/users/cakiki/starred{/owner}{/repo}
https://api.github.com/users/cakiki/subscriptions
User
https://api.github.com/users/cakiki
public
MEMBER
Sure!! And nevermind!
2023-01-19T10:11:38Z
https://github.com/huggingface/datasets/pull/4698#issuecomment-1396729834
https://api.github.com/repos/huggingface/datasets/issues/4698
IC_kwDODunzps5TQGfq
https://api.github.com/repos/huggingface/datasets/issues/comments/1396729834/reactions
2023-01-19T10:11:38Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1396729834
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
Fixes #4637