Dynamic adjusting dataloader sampling weight
MEMBER
You mean based on a condition that has to be checked on-the-fly during training ? Otherwise if you know in advance after how many samples you need to change the sampling you can simply concatenate the two mixes
2025-02-11T10:00:34Z
https://github.com/huggingface/datasets/issues/7387#issuecomment-2650322992
https://api.github.com/repos/huggingface/datasets/issues/7387
IC_kwDODunzps6d-Lgw
https://api.github.com/repos/huggingface/datasets/issues/comments/2650322992/reactions
2025-02-11T10:00:34Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2650322992
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
Yes, like during training, if one data sample's prediction is consistently wrong, its sampling weight gets higher and higher, and if one data sample's prediction is already correct, then we rarely sample it
2025-02-11T10:07:41Z
https://github.com/huggingface/datasets/issues/7387#issuecomment-2650340530
https://api.github.com/repos/huggingface/datasets/issues/7387
IC_kwDODunzps6d-Pyy
https://api.github.com/repos/huggingface/datasets/issues/comments/2650340530/reactions
2025-02-11T10:07:41Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2650340530
https://avatars.githubusercontent.com/u/72799643?v=4
https://api.github.com/users/whc688/events{/privacy}
https://api.github.com/users/whc688/followers
https://api.github.com/users/whc688/following{/other_user}
https://api.github.com/users/whc688/gists{/gist_id}
https://github.com/whc688
whc688
MDQ6VXNlcjcyNzk5NjQz
https://api.github.com/users/whc688/orgs
https://api.github.com/users/whc688/received_events
https://api.github.com/users/whc688/repos
https://api.github.com/users/whc688/starred{/owner}{/repo}
https://api.github.com/users/whc688/subscriptions
User
https://api.github.com/users/whc688
public
MEMBER
it's not possible to use `interleave_datasets()` and modify the probabilities while iterating on the dataset at the moment, so you'd have to implement your own torch `Sampler` or your own`IterableDataset` to implement this logic
2025-02-11T13:24:03Z
https://github.com/huggingface/datasets/issues/7387#issuecomment-2650821758
https://api.github.com/repos/huggingface/datasets/issues/7387
IC_kwDODunzps6eAFR-
https://api.github.com/repos/huggingface/datasets/issues/comments/2650821758/reactions
2025-03-07T14:06:54Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2650821758
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
Hi,
Thanks for your wonderful work! I'm wondering is there a way to dynamically adjust the sampling weight of each data in the dataset during training? Looking forward to your reply, thanks again.