Align IterableDataset.shuffle with Dataset.shuffle
NONE
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_3842). All of your documentation changes will be reflected on that endpoint.
2022-03-07T12:16:57Z
https://github.com/huggingface/datasets/pull/3842#issuecomment-1060626719
https://api.github.com/repos/huggingface/datasets/issues/3842
IC_kwDODunzps4_N-Ef
https://api.github.com/repos/huggingface/datasets/issues/comments/1060626719/reactions
2022-03-07T12:16:57Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1060626719
https://avatars.githubusercontent.com/u/99929124?v=4
https://api.github.com/users/HuggingFaceDocBuilderDev/events{/privacy}
https://api.github.com/users/HuggingFaceDocBuilderDev/followers
https://api.github.com/users/HuggingFaceDocBuilderDev/following{/other_user}
https://api.github.com/users/HuggingFaceDocBuilderDev/gists{/gist_id}
https://github.com/HuggingFaceDocBuilderDev
HuggingFaceDocBuilderDev
U_kgDOBfTMJA
https://api.github.com/users/HuggingFaceDocBuilderDev/orgs
https://api.github.com/users/HuggingFaceDocBuilderDev/received_events
https://api.github.com/users/HuggingFaceDocBuilderDev/repos
https://api.github.com/users/HuggingFaceDocBuilderDev/starred{/owner}{/repo}
https://api.github.com/users/HuggingFaceDocBuilderDev/subscriptions
User
https://api.github.com/users/HuggingFaceDocBuilderDev
public
COLLABORATOR
We should also add `generator` as a param to `shuffle` to fully align the APIs, no?
2022-03-07T13:38:51Z
https://github.com/huggingface/datasets/pull/3842#issuecomment-1060697859
https://api.github.com/repos/huggingface/datasets/issues/3842
IC_kwDODunzps4_OPcD
https://api.github.com/repos/huggingface/datasets/issues/comments/1060697859/reactions
2022-03-07T13:38:51Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1060697859
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
MEMBER
I added the `generator` argument.

I had to make a few other adjustments to make it work. In particular when you call `set_epoch()` on a streaming dataset, it updates the underlying random generator by using a new effective seed. The effective seed is generated using the previous generator and the epoch number.
2022-03-07T16:27:07Z
https://github.com/huggingface/datasets/pull/3842#issuecomment-1060878961
https://api.github.com/repos/huggingface/datasets/issues/3842
IC_kwDODunzps4_O7px
https://api.github.com/repos/huggingface/datasets/issues/comments/1060878961/reactions
2022-03-07T16:27:07Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1060878961
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
From #3444 , Dataset.shuffle can have the same API than IterableDataset.shuffle (i.e. in streaming mode).

Currently you can pass an optional seed to both if you want, BUT currently IterableDataset.shuffle always requires a buffer_size, used for approximate shuffling. I propose using a reasonable default value (maybe 1000) instead.

In this PR, I set the default `buffer_size` value to 1,000, and I reorder the `IterableDataset.shuffle` arguments to match `Dataset.shuffle`, i.e. making `seed` the first argument.