Local paths in common voice
MEMBER
I just changed to `dl_manager.is_streaming` rather than an additional parameter `streaming` that has to be handled by the DatasetBuilder class - this way the streaming logic doesn't interfere with the base builder's code.

I think it's better this way, but let me know if you preferred the previous way and I can revert

> But on the other hand, IMHO, I think this specific solution adds complexity to handling streaming/non-streaming, and moves this complexity to the loading script and thus to the contributors/users who want to create the loading script for their canonical/community datasets (instead of keeping it hidden form the end users).

I'm down to discuss this more in the future !
2022-02-21T16:25:11Z
https://github.com/huggingface/datasets/pull/3736#issuecomment-1047049211
https://api.github.com/repos/huggingface/datasets/issues/3736
IC_kwDODunzps4-aLP7
https://api.github.com/repos/huggingface/datasets/issues/comments/1047049211/reactions
2022-02-21T16:26:22Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1047049211
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
@lhoestq good idea: much cleaner this way! That way each class has its own responsibilities without mixing around...
2022-02-22T07:13:06Z
https://github.com/huggingface/datasets/pull/3736#issuecomment-1047490353
https://api.github.com/repos/huggingface/datasets/issues/3736
IC_kwDODunzps4-b28x
https://api.github.com/repos/huggingface/datasets/issues/comments/1047490353/reactions
2022-02-22T07:13:06Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1047490353
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
Continuation of https://github.com/huggingface/datasets/pull/3664:

- pass the `streaming` parameter to _split_generator
- update @anton-l's code to use this parameter for `common_voice`
- add a comment to explain why we use `download_and_extract` in non-streaming and `iter_archive` in streaming

Now the `common_voice` dataset has a local path back in `ds["path"]`, and this field is `None` in streaming mode.

cc @patrickvonplaten @anton-l @albertvillanova 

Fix #3663.