Add custom fingerprint support to `from_generator`
MEMBER
This is great !

What do you think of passing `config_id=` directly to the builder instead of just the suffix ? This would be a power user argument though, or for internal use. And in from_generator the new argument can be `fingerprint=` as in `Dataset.__init__()`

The `config_id` can be defined using something like `config_id = "default-fingerprint=" + fingerprint`

I feel ike this could make the Dataset API more coherent if we avoid introducing a new argument while we can juste use `fingerprint=`
2025-04-24T10:22:52Z
https://github.com/huggingface/datasets/pull/7533#issuecomment-2827103167
https://api.github.com/repos/huggingface/datasets/issues/7533
IC_kwDODunzps6ogiu_
https://api.github.com/repos/huggingface/datasets/issues/comments/2827103167/reactions
2025-04-24T10:22:52Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2827103167
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
I looked into this issue and the original cause makes total sense — hashing a large generator is clearly inefficient and fragile for big datasets.

PR #7533 looks like a robust and flexible solution! It cleanly separates the fingerprinting responsibility by letting users pass `fingerprint=` (now `config_id=`), which avoids hashing heavy objects like generators but still preserves caching logic.

2025-06-29T06:17:17Z
https://github.com/huggingface/datasets/pull/7533#issuecomment-3016341731
https://api.github.com/repos/huggingface/datasets/issues/7533
IC_kwDODunzps6zybjj
https://api.github.com/repos/huggingface/datasets/issues/comments/3016341731/reactions
2025-06-29T06:17:17Z
https://api.github.com/repos/huggingface/datasets/issues/comments/3016341731
https://avatars.githubusercontent.com/u/142811259?v=4
https://api.github.com/users/ArjunJagdale/events{/privacy}
https://api.github.com/users/ArjunJagdale/followers
https://api.github.com/users/ArjunJagdale/following{/other_user}
https://api.github.com/users/ArjunJagdale/gists{/gist_id}
https://github.com/ArjunJagdale
ArjunJagdale
U_kgDOCIMgew
https://api.github.com/users/ArjunJagdale/orgs
https://api.github.com/users/ArjunJagdale/received_events
https://api.github.com/users/ArjunJagdale/repos
https://api.github.com/users/ArjunJagdale/starred{/owner}{/repo}
https://api.github.com/users/ArjunJagdale/subscriptions
User
https://api.github.com/users/ArjunJagdale
public
NONE
@lhoestq could you please re-review the changes I made?
2025-07-10T09:29:35Z
https://github.com/huggingface/datasets/pull/7533#issuecomment-3056550033
https://api.github.com/repos/huggingface/datasets/issues/7533
IC_kwDODunzps62L0CR
https://api.github.com/repos/huggingface/datasets/issues/comments/3056550033/reactions
2025-07-10T09:29:35Z
https://api.github.com/repos/huggingface/datasets/issues/comments/3056550033
https://avatars.githubusercontent.com/u/43753582?v=4
https://api.github.com/users/simonreise/events{/privacy}
https://api.github.com/users/simonreise/followers
https://api.github.com/users/simonreise/following{/other_user}
https://api.github.com/users/simonreise/gists{/gist_id}
https://github.com/simonreise
simonreise
MDQ6VXNlcjQzNzUzNTgy
https://api.github.com/users/simonreise/orgs
https://api.github.com/users/simonreise/received_events
https://api.github.com/users/simonreise/repos
https://api.github.com/users/simonreise/starred{/owner}{/repo}
https://api.github.com/users/simonreise/subscriptions
User
https://api.github.com/users/simonreise
public
This PR adds `dataset_id_suffix` parameter to 'Dataset.from_generator' function.

`Dataset.from_generator` function passes all of its arguments to `BuilderConfig.create_config_id`, including generator function itself. `BuilderConfig.create_config_id` function tries to hash all the args, which can take a large amount of time or even cause MemoryError if the dataset processed in a generator function is large enough.

This PR allows user to pass a custom fingerprint (`dataset_id_suffix`) to be used as a suffix in a dataset name instead of the one generated by hashing the args.

This PR is a possible solution of #7513 