Perform hidden file check on relative data file path
NONE
_The documentation is not available anymore as the PR was closed or merged._
2022-06-23T14:55:35Z
https://github.com/huggingface/datasets/pull/4551#issuecomment-1164513780
https://api.github.com/repos/huggingface/datasets/issues/4551
IC_kwDODunzps5FaRH0
https://api.github.com/repos/huggingface/datasets/issues/comments/1164513780/reactions
2022-06-30T14:49:20Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1164513780
https://avatars.githubusercontent.com/u/99929124?v=4
https://api.github.com/users/HuggingFaceDocBuilderDev/events{/privacy}
https://api.github.com/users/HuggingFaceDocBuilderDev/followers
https://api.github.com/users/HuggingFaceDocBuilderDev/following{/other_user}
https://api.github.com/users/HuggingFaceDocBuilderDev/gists{/gist_id}
https://github.com/HuggingFaceDocBuilderDev
HuggingFaceDocBuilderDev
U_kgDOBfTMJA
https://api.github.com/users/HuggingFaceDocBuilderDev/orgs
https://api.github.com/users/HuggingFaceDocBuilderDev/received_events
https://api.github.com/users/HuggingFaceDocBuilderDev/repos
https://api.github.com/users/HuggingFaceDocBuilderDev/starred{/owner}{/repo}
https://api.github.com/users/HuggingFaceDocBuilderDev/subscriptions
User
https://api.github.com/users/HuggingFaceDocBuilderDev
public
COLLABORATOR
I'm aware of this behavior, which is tricky to solve due to fsspec's hidden file handling (see https://github.com/huggingface/datasets/issues/4115#issuecomment-1108819538). I've tested some regex patterns to address this, and they seem to work (will push them on Monday; btw they don't break any of fsspec's tests, so maybe we can contribute this as an enhancement to them). Also, perhaps we should include the files starting with `__`  in the results again (we hadn't had issues with this pattern before). WDYT?
2022-06-24T16:37:19Z
https://github.com/huggingface/datasets/pull/4551#issuecomment-1165744178
https://api.github.com/repos/huggingface/datasets/issues/4551
IC_kwDODunzps5Fe9gy
https://api.github.com/repos/huggingface/datasets/issues/comments/1165744178/reactions
2022-06-27T13:18:59Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1165744178
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
MEMBER
I see. Feel free to merge this one if it's good for you btw :)

> Also, perhaps we should include the files starting with __ in the results again (we hadn't had issues with this pattern before)

The point was mainly to ignore `__pycache__` directories for example. Also also for consistency with the iter_files/iter_archive which are already ignoring them
2022-06-28T10:07:28Z
https://github.com/huggingface/datasets/pull/4551#issuecomment-1168517599
https://api.github.com/repos/huggingface/datasets/issues/4551
IC_kwDODunzps5Fpinf
https://api.github.com/repos/huggingface/datasets/issues/comments/1168517599/reactions
2022-06-28T17:29:24Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1168517599
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
COLLABORATOR
Very elegant solution! Feel free to merge if the CI is green after adding the tests.
2022-06-30T13:14:34Z
https://github.com/huggingface/datasets/pull/4551#issuecomment-1171205249
https://api.github.com/repos/huggingface/datasets/issues/4551
IC_kwDODunzps5FzyyB
https://api.github.com/repos/huggingface/datasets/issues/comments/1171205249/reactions
2022-06-30T13:14:34Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1171205249
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
MEMBER
CI failure is unrelated to this PR
2022-06-30T14:38:37Z
https://github.com/huggingface/datasets/pull/4551#issuecomment-1171303740
https://api.github.com/repos/huggingface/datasets/issues/4551
IC_kwDODunzps5F0K08
https://api.github.com/repos/huggingface/datasets/issues/comments/1171303740/reactions
2022-06-30T14:38:37Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1171303740
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
Fix #4549 