Make iter_archive work with ZIP files
CONTRIBUTOR
Hello, is this issue open for any contributor ? can I work on it ?


2021-11-22T20:53:29Z
https://github.com/huggingface/datasets/issues/3272#issuecomment-975909023
https://api.github.com/repos/huggingface/datasets/issues/3272
IC_kwDODunzps46KzCf
https://api.github.com/repos/huggingface/datasets/issues/comments/975909023/reactions
2021-11-22T20:53:29Z
https://api.github.com/repos/huggingface/datasets/issues/comments/975909023
https://avatars.githubusercontent.com/u/56029953?v=4
https://api.github.com/users/Mehdi2402/events{/privacy}
https://api.github.com/users/Mehdi2402/followers
https://api.github.com/users/Mehdi2402/following{/other_user}
https://api.github.com/users/Mehdi2402/gists{/gist_id}
https://github.com/Mehdi2402
Mehdi2402
MDQ6VXNlcjU2MDI5OTUz
https://api.github.com/users/Mehdi2402/orgs
https://api.github.com/users/Mehdi2402/received_events
https://api.github.com/users/Mehdi2402/repos
https://api.github.com/users/Mehdi2402/starred{/owner}{/repo}
https://api.github.com/users/Mehdi2402/subscriptions
User
https://api.github.com/users/Mehdi2402
public
MEMBER
Hi ! Sure this is open for any contributor. If you're interested feel free to self-assign this issue to you by commenting `#self-assign`. Then if you have any question or if I can help, feel free to ping me.

To begin with, feel free to take a look at both implementations of `iter_archive` for local downloads and for data streaming:

In the `DownloadManager` for local dowloads:
https://github.com/huggingface/datasets/blob/dfa334bd8dc6cbc854b170379c7d2cb7e3d3fe4f/src/datasets/utils/download_manager.py#L218-L242

In the `StreamingDownloadManager` to stream the content of the archive directly from the remote file:
https://github.com/huggingface/datasets/blob/dfa334bd8dc6cbc854b170379c7d2cb7e3d3fe4f/src/datasets/utils/streaming_download_manager.py#L502-L526

Notice the call to `xopen` that opens and streams a file given either an URL or a local path :)
2021-11-24T17:38:32Z
https://github.com/huggingface/datasets/issues/3272#issuecomment-978091401
https://api.github.com/repos/huggingface/datasets/issues/3272
IC_kwDODunzps46TH2J
https://api.github.com/repos/huggingface/datasets/issues/comments/978091401/reactions
2021-11-24T17:39:03Z
https://api.github.com/repos/huggingface/datasets/issues/comments/978091401
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Okay thank you for the information. I will work on this :) 
2021-11-25T00:06:37Z
https://github.com/huggingface/datasets/issues/3272#issuecomment-978514266
https://api.github.com/repos/huggingface/datasets/issues/3272
IC_kwDODunzps46UvFa
https://api.github.com/repos/huggingface/datasets/issues/comments/978514266/reactions
2021-11-25T00:07:53Z
https://api.github.com/repos/huggingface/datasets/issues/comments/978514266
https://avatars.githubusercontent.com/u/56029953?v=4
https://api.github.com/users/Mehdi2402/events{/privacy}
https://api.github.com/users/Mehdi2402/followers
https://api.github.com/users/Mehdi2402/following{/other_user}
https://api.github.com/users/Mehdi2402/gists{/gist_id}
https://github.com/Mehdi2402
Mehdi2402
MDQ6VXNlcjU2MDI5OTUz
https://api.github.com/users/Mehdi2402/orgs
https://api.github.com/users/Mehdi2402/received_events
https://api.github.com/users/Mehdi2402/repos
https://api.github.com/users/Mehdi2402/starred{/owner}{/repo}
https://api.github.com/users/Mehdi2402/subscriptions
User
https://api.github.com/users/Mehdi2402
public
CONTRIBUTOR
#self-assign
2021-11-25T00:08:35Z
https://github.com/huggingface/datasets/issues/3272#issuecomment-978517417
https://api.github.com/repos/huggingface/datasets/issues/3272
IC_kwDODunzps46Uv2p
https://api.github.com/repos/huggingface/datasets/issues/comments/978517417/reactions
2021-11-25T00:08:35Z
https://api.github.com/repos/huggingface/datasets/issues/comments/978517417
https://avatars.githubusercontent.com/u/56029953?v=4
https://api.github.com/users/Mehdi2402/events{/privacy}
https://api.github.com/users/Mehdi2402/followers
https://api.github.com/users/Mehdi2402/following{/other_user}
https://api.github.com/users/Mehdi2402/gists{/gist_id}
https://github.com/Mehdi2402
Mehdi2402
MDQ6VXNlcjU2MDI5OTUz
https://api.github.com/users/Mehdi2402/orgs
https://api.github.com/users/Mehdi2402/received_events
https://api.github.com/users/Mehdi2402/repos
https://api.github.com/users/Mehdi2402/starred{/owner}{/repo}
https://api.github.com/users/Mehdi2402/subscriptions
User
https://api.github.com/users/Mehdi2402
public
Currently users can use `dl_manager.iter_archive` in their dataset script to iterate over all the files of a TAR archive.
It would be nice if it could work with ZIP files too !