Support streaming tar files
MEMBER
Hi ! Why do we need the custom `readline` for exactly ? feel free to add a comment to say why it's needed
2021-08-26T10:02:30Z
https://github.com/huggingface/datasets/pull/2800#issuecomment-906267282
https://api.github.com/repos/huggingface/datasets/issues/2800
IC_kwDODunzps42BIqS
https://api.github.com/repos/huggingface/datasets/issues/comments/906267282/reactions
2021-08-26T10:02:30Z
https://api.github.com/repos/huggingface/datasets/issues/comments/906267282
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
This PR adds support to stream tar files by using the `fsspec` tar protocol.

It also uses the custom `readline` implemented in PR #2786.

The corresponding test is implemented in PR #2786.