Register `.zstd` extension for zstd-compressed files
NONE
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_7032). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
2024-07-08T12:42:16Z
https://github.com/huggingface/datasets/pull/7032#issuecomment-2213973855
https://api.github.com/repos/huggingface/datasets/issues/7032
IC_kwDODunzps6D9o9f
https://api.github.com/repos/huggingface/datasets/issues/comments/2213973855/reactions
2024-07-08T12:42:16Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2213973855
https://avatars.githubusercontent.com/u/99929124?v=4
https://api.github.com/users/HuggingFaceDocBuilderDev/events{/privacy}
https://api.github.com/users/HuggingFaceDocBuilderDev/followers
https://api.github.com/users/HuggingFaceDocBuilderDev/following{/other_user}
https://api.github.com/users/HuggingFaceDocBuilderDev/gists{/gist_id}
https://github.com/HuggingFaceDocBuilderDev
HuggingFaceDocBuilderDev
U_kgDOBfTMJA
https://api.github.com/users/HuggingFaceDocBuilderDev/orgs
https://api.github.com/users/HuggingFaceDocBuilderDev/received_events
https://api.github.com/users/HuggingFaceDocBuilderDev/repos
https://api.github.com/users/HuggingFaceDocBuilderDev/starred{/owner}{/repo}
https://api.github.com/users/HuggingFaceDocBuilderDev/subscriptions
User
https://api.github.com/users/HuggingFaceDocBuilderDev
public
CONTRIBUTOR
@albertvillanova hm I don't know tbh, it's just that "mlfoundations/dclm-baseline-1.0" dataset contains [files](https://huggingface.co/datasets/mlfoundations/dclm-baseline-1.0/tree/main/global-shard_01_of_10/local-shard_0_of_10) with this extension and these files seem to be valid 
2024-07-09T14:08:42Z
https://github.com/huggingface/datasets/pull/7032#issuecomment-2217843743
https://api.github.com/repos/huggingface/datasets/issues/7032
IC_kwDODunzps6EMZwf
https://api.github.com/repos/huggingface/datasets/issues/comments/2217843743/reactions
2024-07-09T14:08:42Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2217843743
https://avatars.githubusercontent.com/u/16348744?v=4
https://api.github.com/users/polinaeterna/events{/privacy}
https://api.github.com/users/polinaeterna/followers
https://api.github.com/users/polinaeterna/following{/other_user}
https://api.github.com/users/polinaeterna/gists{/gist_id}
https://github.com/polinaeterna
polinaeterna
MDQ6VXNlcjE2MzQ4NzQ0
https://api.github.com/users/polinaeterna/orgs
https://api.github.com/users/polinaeterna/received_events
https://api.github.com/users/polinaeterna/repos
https://api.github.com/users/polinaeterna/starred{/owner}{/repo}
https://api.github.com/users/polinaeterna/subscriptions
User
https://api.github.com/users/polinaeterna
public
CONTRIBUTOR
not sure why CI is failing but seems to be unrelated to this pr? can I merge @lhoestq @albertvillanova ?
2024-07-09T16:34:40Z
https://github.com/huggingface/datasets/pull/7032#issuecomment-2218153806
https://api.github.com/repos/huggingface/datasets/issues/7032
IC_kwDODunzps6ENldO
https://api.github.com/repos/huggingface/datasets/issues/comments/2218153806/reactions
2024-07-09T16:34:40Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2218153806
https://avatars.githubusercontent.com/u/16348744?v=4
https://api.github.com/users/polinaeterna/events{/privacy}
https://api.github.com/users/polinaeterna/followers
https://api.github.com/users/polinaeterna/following{/other_user}
https://api.github.com/users/polinaeterna/gists{/gist_id}
https://github.com/polinaeterna
polinaeterna
MDQ6VXNlcjE2MzQ4NzQ0
https://api.github.com/users/polinaeterna/orgs
https://api.github.com/users/polinaeterna/received_events
https://api.github.com/users/polinaeterna/repos
https://api.github.com/users/polinaeterna/starred{/owner}{/repo}
https://api.github.com/users/polinaeterna/subscriptions
User
https://api.github.com/users/polinaeterna
public
MEMBER
yes you can merge, the CI failure is unrelated (surely an issue with hub-ci)
2024-07-09T18:18:11Z
https://github.com/huggingface/datasets/pull/7032#issuecomment-2218362826
https://api.github.com/repos/huggingface/datasets/issues/7032
IC_kwDODunzps6EOYfK
https://api.github.com/repos/huggingface/datasets/issues/comments/2218362826/reactions
2024-07-09T18:18:11Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2218362826
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
ah why not, you could try opening a PR

btw there is a channel with them at (internal) https://app.slack.com/client/T1RCG4490/C079AKTV11P if you want to let them know
2024-07-10T14:13:47Z
https://github.com/huggingface/datasets/pull/7032#issuecomment-2220629214
https://api.github.com/repos/huggingface/datasets/issues/7032
IC_kwDODunzps6EXBze
https://api.github.com/repos/huggingface/datasets/issues/comments/2220629214/reactions
2024-07-10T14:13:47Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2220629214
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
@lhoestq, your previous comment was addressed to me or Polina?

@polinaeterna let me know if it is OK for you.
2024-07-11T08:15:15Z
https://github.com/huggingface/datasets/pull/7032#issuecomment-2222313393
https://api.github.com/repos/huggingface/datasets/issues/7032
IC_kwDODunzps6Edc-x
https://api.github.com/repos/huggingface/datasets/issues/comments/2222313393/reactions
2024-07-11T08:15:15Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2222313393
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
MEMBER
I opened https://huggingface.co/datasets/mlfoundations/dclm-baseline-1.0/discussions/7
2024-07-11T15:32:56Z
https://github.com/huggingface/datasets/pull/7032#issuecomment-2223257494
https://api.github.com/repos/huggingface/datasets/issues/7032
IC_kwDODunzps6EhDeW
https://api.github.com/repos/huggingface/datasets/issues/comments/2223257494/reactions
2024-07-11T15:32:56Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2223257494
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
Should we close this PR then?
2024-07-12T13:50:38Z
https://github.com/huggingface/datasets/pull/7032#issuecomment-2225632876
https://api.github.com/repos/huggingface/datasets/issues/7032
IC_kwDODunzps6EqHZs
https://api.github.com/repos/huggingface/datasets/issues/comments/2225632876/reactions
2024-07-12T13:50:38Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2225632876
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
For example, https://huggingface.co/datasets/mlfoundations/dclm-baseline-1.0 dataset files have `.zstd` extension which is currently ignored (only `.zst` is registered).