Download datasets from a private hub
NONE
For reference:
https://github.com/huggingface/transformers/issues/15514
https://github.com/huggingface/huggingface_hub/issues/650
2022-02-04T11:00:16Z
https://github.com/huggingface/datasets/issues/3679#issuecomment-1029882386
https://api.github.com/repos/huggingface/datasets/issues/3679
IC_kwDODunzps49YsIS
https://api.github.com/repos/huggingface/datasets/issues/comments/1029882386/reactions
2022-02-04T11:00:16Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1029882386
https://avatars.githubusercontent.com/u/3436143?v=4
https://api.github.com/users/juliensimon/events{/privacy}
https://api.github.com/users/juliensimon/followers
https://api.github.com/users/juliensimon/following{/other_user}
https://api.github.com/users/juliensimon/gists{/gist_id}
https://github.com/juliensimon
juliensimon
MDQ6VXNlcjM0MzYxNDM=
https://api.github.com/users/juliensimon/orgs
https://api.github.com/users/juliensimon/received_events
https://api.github.com/users/juliensimon/repos
https://api.github.com/users/juliensimon/starred{/owner}{/repo}
https://api.github.com/users/juliensimon/subscriptions
User
https://api.github.com/users/juliensimon
public
MEMBER
Hi ! For information one can set the environment variable `HF_ENDPOINT` (default is `https://huggingface.co`) if they want to use a private hub.

We may need to coordinate with the other libraries to have a consistent way of changing the hub endpoint
2022-02-04T13:24:44Z
https://github.com/huggingface/datasets/issues/3679#issuecomment-1029983140
https://api.github.com/repos/huggingface/datasets/issues/3679
IC_kwDODunzps49ZEuk
https://api.github.com/repos/huggingface/datasets/issues/comments/1029983140/reactions
2022-02-04T13:24:44Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1029983140
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
Yes, I tested it successfully this morning. Thanks.
2022-02-22T11:08:07Z
https://github.com/huggingface/datasets/issues/3679#issuecomment-1047683451
https://api.github.com/repos/huggingface/datasets/issues/3679
IC_kwDODunzps4-cmF7
https://api.github.com/repos/huggingface/datasets/issues/comments/1047683451/reactions
2022-02-22T11:08:07Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1047683451
https://avatars.githubusercontent.com/u/3436143?v=4
https://api.github.com/users/juliensimon/events{/privacy}
https://api.github.com/users/juliensimon/followers
https://api.github.com/users/juliensimon/following{/other_user}
https://api.github.com/users/juliensimon/gists{/gist_id}
https://github.com/juliensimon
juliensimon
MDQ6VXNlcjM0MzYxNDM=
https://api.github.com/users/juliensimon/orgs
https://api.github.com/users/juliensimon/received_events
https://api.github.com/users/juliensimon/repos
https://api.github.com/users/juliensimon/starred{/owner}{/repo}
https://api.github.com/users/juliensimon/subscriptions
User
https://api.github.com/users/juliensimon
public
In the context of a private hub deployment, customers would like to use load_dataset() to load datasets from their hub, not from the public hub. This doesn't seem to be configurable at the moment and it would be nice to add this feature.

The obvious workaround is to clone the repo first and then load it from local storage, but this adds an extra step. It'd be great to have the same experience regardless of where the hub is hosted.

The same issue exists with the transformers library and the CLI. I'm going to create issues there as well, and I'll reference them below.