Add repeat() for iterable datasets
CONTRIBUTOR
perhaps concatenate_datasets can already be used to achieve almost the same effect? 
2024-10-03T09:59:16Z
https://github.com/huggingface/datasets/issues/7192#issuecomment-2391004769
https://api.github.com/repos/huggingface/datasets/issues/7192
IC_kwDODunzps6Og9Zh
https://api.github.com/repos/huggingface/datasets/issues/comments/2391004769/reactions
2024-10-03T09:59:16Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2391004769
https://avatars.githubusercontent.com/u/5719745?v=4
https://api.github.com/users/alex-hh/events{/privacy}
https://api.github.com/users/alex-hh/followers
https://api.github.com/users/alex-hh/following{/other_user}
https://api.github.com/users/alex-hh/gists{/gist_id}
https://github.com/alex-hh
alex-hh
MDQ6VXNlcjU3MTk3NDU=
https://api.github.com/users/alex-hh/orgs
https://api.github.com/users/alex-hh/received_events
https://api.github.com/users/alex-hh/repos
https://api.github.com/users/alex-hh/starred{/owner}{/repo}
https://api.github.com/users/alex-hh/subscriptions
User
https://api.github.com/users/alex-hh
public
MEMBER
`concatenate_datasets` does the job when there is a finite number of repetitions, but in case of `.repeat()` forever we need a new logic in `iterable_dataset.py`
2024-10-03T12:53:33Z
https://github.com/huggingface/datasets/issues/7192#issuecomment-2391348186
https://api.github.com/repos/huggingface/datasets/issues/7192
IC_kwDODunzps6OiRPa
https://api.github.com/repos/huggingface/datasets/issues/comments/2391348186/reactions
2024-10-03T12:53:33Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2391348186
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
done in https://github.com/huggingface/datasets/pull/7198
2025-03-18T10:48:32Z
https://github.com/huggingface/datasets/issues/7192#issuecomment-2732660956
https://api.github.com/repos/huggingface/datasets/issues/7192
IC_kwDODunzps6i4Rjc
https://api.github.com/repos/huggingface/datasets/issues/comments/2732660956/reactions
2025-03-18T10:48:32Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2732660956
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
### Feature request

It would be useful to be able to straightforwardly repeat iterable datasets indefinitely, to provide complete control over starting and ending of iteration to the user.

An IterableDataset.repeat(n) function could do this automatically

### Motivation

This feature was discussed in this issue https://github.com/huggingface/datasets/issues/7147, and would resolve the need to use the hack of interleave datasets with probability 0 as a simple way to achieve this functionality.

An additional benefit might be the simplification of the use of iterable datasets in a distributed setting:
If the user can assume that datasets will repeat indefinitely, then issues around different numbers of samples appearing on different devices (e.g. https://github.com/huggingface/datasets/issues/6437, https://github.com/huggingface/datasets/issues/6594, https://github.com/huggingface/datasets/issues/6623, https://github.com/huggingface/datasets/issues/6719) can potentially be straightforwardly resolved by simply doing:

ids.repeat(None).take(n_samples_per_epoch)


### Your contribution

I'm not familiar enough with the codebase to assess how straightforward this would be to implement.

If it might be very straightforward, I could possibly have a go.