Adding option for multipart achive download
NONE
On closer examination, this appears to be unnecessary. 
2024-02-25T08:13:01Z
https://github.com/huggingface/datasets/pull/6572#issuecomment-1962851825
https://api.github.com/repos/huggingface/datasets/issues/6572
IC_kwDODunzps50_r3x
https://api.github.com/repos/huggingface/datasets/issues/comments/1962851825/reactions
2024-02-25T08:13:01Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1962851825
https://avatars.githubusercontent.com/u/66251151?v=4
https://api.github.com/users/jpodivin/events{/privacy}
https://api.github.com/users/jpodivin/followers
https://api.github.com/users/jpodivin/following{/other_user}
https://api.github.com/users/jpodivin/gists{/gist_id}
https://github.com/jpodivin
jpodivin
MDQ6VXNlcjY2MjUxMTUx
https://api.github.com/users/jpodivin/orgs
https://api.github.com/users/jpodivin/received_events
https://api.github.com/users/jpodivin/repos
https://api.github.com/users/jpodivin/starred{/owner}{/repo}
https://api.github.com/users/jpodivin/subscriptions
User
https://api.github.com/users/jpodivin
public
Right now we can only download multiple separate archives or a single file archive, but not multipart archives, such as those produced by `tar --multi-volume`. This PR allows for downloading and extraction of archives split into multiple parts.

With the new `multi_part` field of the `DownloadConfig` set, the downloader will first retrieve all the files and attempt to concatenate them before starting extraction. This will obviously fail if files retrieved are actually multiple separate archives, so the option is set to `False` by default.

Tests and docs incoming. 