Stream private zipped images
NONE
_The documentation is not available anymore as the PR was closed or merged._
2022-04-14T15:36:03Z
https://github.com/huggingface/datasets/pull/4173#issuecomment-1099309583
https://api.github.com/repos/huggingface/datasets/issues/4173
IC_kwDODunzps5BhiIP
https://api.github.com/repos/huggingface/datasets/issues/comments/1099309583/reactions
2022-05-05T14:05:54Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1099309583
https://avatars.githubusercontent.com/u/99929124?v=4
https://api.github.com/users/HuggingFaceDocBuilderDev/events{/privacy}
https://api.github.com/users/HuggingFaceDocBuilderDev/followers
https://api.github.com/users/HuggingFaceDocBuilderDev/following{/other_user}
https://api.github.com/users/HuggingFaceDocBuilderDev/gists{/gist_id}
https://github.com/HuggingFaceDocBuilderDev
HuggingFaceDocBuilderDev
U_kgDOBfTMJA
https://api.github.com/users/HuggingFaceDocBuilderDev/orgs
https://api.github.com/users/HuggingFaceDocBuilderDev/received_events
https://api.github.com/users/HuggingFaceDocBuilderDev/repos
https://api.github.com/users/HuggingFaceDocBuilderDev/starred{/owner}{/repo}
https://api.github.com/users/HuggingFaceDocBuilderDev/subscriptions
User
https://api.github.com/users/HuggingFaceDocBuilderDev
public
MEMBER
oops looks like some tests are failing sorry, will fix them tomorrow

EDIT: not today but asap hopefully
2022-04-14T16:15:29Z
https://github.com/huggingface/datasets/pull/4173#issuecomment-1099358391
https://api.github.com/repos/huggingface/datasets/issues/4173
IC_kwDODunzps5BhuC3
https://api.github.com/repos/huggingface/datasets/issues/comments/1099358391/reactions
2022-04-19T15:29:41Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1099358391
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
cc @mariosasko this is ready for review, let me know what you think !
2022-05-03T10:23:56Z
https://github.com/huggingface/datasets/pull/4173#issuecomment-1115946249
https://api.github.com/repos/huggingface/datasets/issues/4173
IC_kwDODunzps5Cg_0J
https://api.github.com/repos/huggingface/datasets/issues/comments/1115946249/reactions
2022-05-03T10:23:56Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1115946249
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
As mentioned in https://github.com/huggingface/datasets/issues/4139 it's currently not possible to stream private/gated zipped images from the Hub.

This is because `Image.decode_example` does not handle authentication. Indeed decoding requires to access and download the file from the private repository.

In this PR I added authentication to `Image.decode_example` via a `token_per_repo_id` optional argument. I first wanted to just pass `use_auth_token` but a single `Image` instance can be responsible of decoding images from a combination of several datasets together (from `interleave_datasets` for example). Therefore I just used a dictionary `repo_id` -> `token` instead.

I'm getting the `repo_id` from the dataset builder (I replaced the `namepace` attribute with `repo_id`)

I did the same for `Audio.decode_example`

cc @SBrandeis @severo 