Add repeat method to datasets
CONTRIBUTOR
@lhoestq does this look reasonable?
2025-01-28T09:41:53Z
https://github.com/huggingface/datasets/pull/7198#issuecomment-2618443145
https://api.github.com/repos/huggingface/datasets/issues/7198
IC_kwDODunzps6cEkWJ
https://api.github.com/repos/huggingface/datasets/issues/comments/2618443145/reactions
2025-01-28T09:41:53Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2618443145
https://avatars.githubusercontent.com/u/5719745?v=4
https://api.github.com/users/alex-hh/events{/privacy}
https://api.github.com/users/alex-hh/followers
https://api.github.com/users/alex-hh/following{/other_user}
https://api.github.com/users/alex-hh/gists{/gist_id}
https://github.com/alex-hh
alex-hh
MDQ6VXNlcjU3MTk3NDU=
https://api.github.com/users/alex-hh/orgs
https://api.github.com/users/alex-hh/received_events
https://api.github.com/users/alex-hh/repos
https://api.github.com/users/alex-hh/starred{/owner}{/repo}
https://api.github.com/users/alex-hh/subscriptions
User
https://api.github.com/users/alex-hh
public
CONTRIBUTOR
updated and added test cases!
2025-01-29T22:14:36Z
https://github.com/huggingface/datasets/pull/7198#issuecomment-2622980293
https://api.github.com/repos/huggingface/datasets/issues/7198
IC_kwDODunzps6cV4DF
https://api.github.com/repos/huggingface/datasets/issues/comments/2622980293/reactions
2025-01-29T22:14:36Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2622980293
https://avatars.githubusercontent.com/u/5719745?v=4
https://api.github.com/users/alex-hh/events{/privacy}
https://api.github.com/users/alex-hh/followers
https://api.github.com/users/alex-hh/following{/other_user}
https://api.github.com/users/alex-hh/gists{/gist_id}
https://github.com/alex-hh
alex-hh
MDQ6VXNlcjU3MTk3NDU=
https://api.github.com/users/alex-hh/orgs
https://api.github.com/users/alex-hh/received_events
https://api.github.com/users/alex-hh/repos
https://api.github.com/users/alex-hh/starred{/owner}{/repo}
https://api.github.com/users/alex-hh/subscriptions
User
https://api.github.com/users/alex-hh
public
NONE
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_7198). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
2025-01-30T10:52:02Z
https://github.com/huggingface/datasets/pull/7198#issuecomment-2624151120
https://api.github.com/repos/huggingface/datasets/issues/7198
IC_kwDODunzps6caV5Q
https://api.github.com/repos/huggingface/datasets/issues/comments/2624151120/reactions
2025-01-30T10:52:02Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2624151120
https://avatars.githubusercontent.com/u/99929124?v=4
https://api.github.com/users/HuggingFaceDocBuilderDev/events{/privacy}
https://api.github.com/users/HuggingFaceDocBuilderDev/followers
https://api.github.com/users/HuggingFaceDocBuilderDev/following{/other_user}
https://api.github.com/users/HuggingFaceDocBuilderDev/gists{/gist_id}
https://github.com/HuggingFaceDocBuilderDev
HuggingFaceDocBuilderDev
U_kgDOBfTMJA
https://api.github.com/users/HuggingFaceDocBuilderDev/orgs
https://api.github.com/users/HuggingFaceDocBuilderDev/received_events
https://api.github.com/users/HuggingFaceDocBuilderDev/repos
https://api.github.com/users/HuggingFaceDocBuilderDev/starred{/owner}{/repo}
https://api.github.com/users/HuggingFaceDocBuilderDev/subscriptions
User
https://api.github.com/users/HuggingFaceDocBuilderDev
public
CONTRIBUTOR
thanks for the fixes!
2025-02-05T15:48:27Z
https://github.com/huggingface/datasets/pull/7198#issuecomment-2637302341
https://api.github.com/repos/huggingface/datasets/issues/7198
IC_kwDODunzps6dMgpF
https://api.github.com/repos/huggingface/datasets/issues/comments/2637302341/reactions
2025-02-05T15:48:27Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2637302341
https://avatars.githubusercontent.com/u/5719745?v=4
https://api.github.com/users/alex-hh/events{/privacy}
https://api.github.com/users/alex-hh/followers
https://api.github.com/users/alex-hh/following{/other_user}
https://api.github.com/users/alex-hh/gists{/gist_id}
https://github.com/alex-hh
alex-hh
MDQ6VXNlcjU3MTk3NDU=
https://api.github.com/users/alex-hh/orgs
https://api.github.com/users/alex-hh/received_events
https://api.github.com/users/alex-hh/repos
https://api.github.com/users/alex-hh/starred{/owner}{/repo}
https://api.github.com/users/alex-hh/subscriptions
User
https://api.github.com/users/alex-hh
public
Following up on discussion in #6623 and #7198 I thought this would be pretty useful for my case so had a go at implementing.

My main motivation is to be able to call iterable_dataset.repeat(None).take(samples_per_epoch) to safely avoid timeout issues in a distributed training setting. This would provide a straightforward workaround for several open issues related to this situation:  https://github.com/huggingface/datasets/issues/6437, https://github.com/huggingface/datasets/issues/6594, https://github.com/huggingface/datasets/issues/6623, https://github.com/huggingface/datasets/issues/6719.

@lhoestq let me know if this looks on the right track!