Add `Dataset.from_generator`
MEMBER
I restarted the builder PR job just in case
2022-09-13T10:17:16Z
https://github.com/huggingface/datasets/pull/4957#issuecomment-1245201105
https://api.github.com/repos/huggingface/datasets/issues/4957
IC_kwDODunzps5KOELR
https://api.github.com/repos/huggingface/datasets/issues/comments/1245201105/reactions
2022-09-13T10:17:16Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1245201105
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
_The documentation is not available anymore as the PR was closed or merged._
2022-09-13T16:58:36Z
https://github.com/huggingface/datasets/pull/4957#issuecomment-1245689668
https://api.github.com/repos/huggingface/datasets/issues/4957
IC_kwDODunzps5KP7dE
https://api.github.com/repos/huggingface/datasets/issues/comments/1245689668/reactions
2022-09-16T14:46:35Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1245689668
https://avatars.githubusercontent.com/u/99929124?v=4
https://api.github.com/users/HuggingFaceDocBuilderDev/events{/privacy}
https://api.github.com/users/HuggingFaceDocBuilderDev/followers
https://api.github.com/users/HuggingFaceDocBuilderDev/following{/other_user}
https://api.github.com/users/HuggingFaceDocBuilderDev/gists{/gist_id}
https://github.com/HuggingFaceDocBuilderDev
HuggingFaceDocBuilderDev
U_kgDOBfTMJA
https://api.github.com/users/HuggingFaceDocBuilderDev/orgs
https://api.github.com/users/HuggingFaceDocBuilderDev/received_events
https://api.github.com/users/HuggingFaceDocBuilderDev/repos
https://api.github.com/users/HuggingFaceDocBuilderDev/starred{/owner}{/repo}
https://api.github.com/users/HuggingFaceDocBuilderDev/subscriptions
User
https://api.github.com/users/HuggingFaceDocBuilderDev
public
COLLABORATOR
CI is now green. https://github.com/huggingface/doc-builder/pull/296 explains why it failed.
2022-09-14T13:35:49Z
https://github.com/huggingface/datasets/pull/4957#issuecomment-1246776259
https://api.github.com/repos/huggingface/datasets/issues/4957
IC_kwDODunzps5KUEvD
https://api.github.com/repos/huggingface/datasets/issues/comments/1246776259/reactions
2022-09-14T13:35:49Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1246776259
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
Add `Dataset.from_generator` to the API to allow creating datasets from data larger than RAM. The implementation relies on a packaged module not exposed in `load_dataset` to tie this method with `datasets`' caching mechanism.

Closes https://github.com/huggingface/datasets/issues/4417