ImageFolder add option to ignore some folders like '.ipynb_checkpoints'
CONTRIBUTOR
Maybe it would be nice to ignore private dirs like this one (ones starting with `.`) by default. 

CC @mariosasko 
2022-04-06T17:31:40Z
https://github.com/huggingface/datasets/issues/4115#issuecomment-1090529069
https://api.github.com/repos/huggingface/datasets/issues/4115
IC_kwDODunzps5BACct
https://api.github.com/repos/huggingface/datasets/issues/comments/1090529069/reactions
2022-04-06T17:31:40Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1090529069
https://avatars.githubusercontent.com/u/32437151?v=4
https://api.github.com/users/nateraw/events{/privacy}
https://api.github.com/users/nateraw/followers
https://api.github.com/users/nateraw/following{/other_user}
https://api.github.com/users/nateraw/gists{/gist_id}
https://github.com/nateraw
nateraw
MDQ6VXNlcjMyNDM3MTUx
https://api.github.com/users/nateraw/orgs
https://api.github.com/users/nateraw/received_events
https://api.github.com/users/nateraw/repos
https://api.github.com/users/nateraw/starred{/owner}{/repo}
https://api.github.com/users/nateraw/subscriptions
User
https://api.github.com/users/nateraw
public
COLLABORATOR
Maybe we can add a `ignore_hidden_files` flag to the builder configs of our packaged loaders (to be consistent across all of them), wdyt @lhoestq @albertvillanova? 
2022-04-21T17:08:55Z
https://github.com/huggingface/datasets/issues/4115#issuecomment-1105479594
https://api.github.com/repos/huggingface/datasets/issues/4115
IC_kwDODunzps5B5Eeq
https://api.github.com/repos/huggingface/datasets/issues/comments/1105479594/reactions
2022-04-21T17:08:55Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1105479594
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
MEMBER
I think they should always ignore them actually ! Not sure if adding a flag would be helpful
2022-04-22T09:48:49Z
https://github.com/huggingface/datasets/issues/4115#issuecomment-1106281858
https://api.github.com/repos/huggingface/datasets/issues/4115
IC_kwDODunzps5B8IWC
https://api.github.com/repos/huggingface/datasets/issues/comments/1106281858/reactions
2022-04-22T09:48:49Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1106281858
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
COLLABORATOR
@lhoestq But what if the user explicitly requests those files via regex?

`glob.glob` ignores hidden files (files starting with ".") by default unless they are explicitly requested, but fsspec's `glob` doesn't follow this behavior, which is probably a bug, so maybe we can raise an issue or open a PR in their repo?
2022-04-25T17:01:41Z
https://github.com/huggingface/datasets/issues/4115#issuecomment-1108819538
https://api.github.com/repos/huggingface/datasets/issues/4115
IC_kwDODunzps5CFz5S
https://api.github.com/repos/huggingface/datasets/issues/comments/1108819538/reactions
2022-04-25T17:01:41Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1108819538
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
MEMBER
> @lhoestq But what if the user explicitly requests those files via regex?

Usually hidden files are meant to be ignored. If they are data files, they must be placed outside a hidden directory in the first place right ? I think it's more sensible to explain this than adding a flag.

> glob.glob ignores hidden files (files starting with ".") by default unless they are explicitly requested, but fsspec's glob doesn't follow this behavior, which is probably a bug, so maybe we can raise an issue or open a PR in their repo?

After globbing using `fsspec`, we already ignore files that start with a `.` in `_resolve_single_pattern_locally` and `_resolve_single_pattern_in_dataset_repository`, I guess we can just account for parent directories as well ?

We could open an issue on `fsspec` but I think they won't change this since it's an important breaking change for them.
2022-04-26T16:39:42Z
https://github.com/huggingface/datasets/issues/4115#issuecomment-1110014182
https://api.github.com/repos/huggingface/datasets/issues/4115
IC_kwDODunzps5CKXjm
https://api.github.com/repos/huggingface/datasets/issues/comments/1110014182/reactions
2022-04-26T16:39:42Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1110014182
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
**Is your feature request related to a problem? Please describe.**
I sometimes like to peek at the dataset images from jupyterlab. thus '.ipynb_checkpoints' folder appears where my dataset is and (just realized) leads to accidental duplicate image additions. I think this is an easy enough thing to miss especially if the dataset is very large.

**Describe the solution you'd like**
maybe have an option `ignore` or something .gitignore style
`dataset = load_dataset("imagefolder", data_dir="./data/original", ignore="regex?")`

**Describe alternatives you've considered**
Could filter out manually

