Use `iter_files` instead of `str(Path(...)` in image dataset
MEMBER
`iter_archive` is about to support ZIP archives. I think we should use this no ?

see #3347 https://github.com/huggingface/datasets/pull/3379
2021-12-27T16:07:16Z
https://github.com/huggingface/datasets/pull/3477#issuecomment-1001636004
https://api.github.com/repos/huggingface/datasets/issues/3477
IC_kwDODunzps47s8Ck
https://api.github.com/repos/huggingface/datasets/issues/comments/1001636004/reactions
2021-12-27T16:07:51Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1001636004
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
I was interested in the support for isfile/dir in remote.

Anyway, `iter_files` will be available for community users.
2021-12-27T16:11:03Z
https://github.com/huggingface/datasets/pull/3477#issuecomment-1001637665
https://api.github.com/repos/huggingface/datasets/issues/3477
IC_kwDODunzps47s8ch
https://api.github.com/repos/huggingface/datasets/issues/comments/1001637665/reactions
2021-12-27T16:11:03Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1001637665
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
MEMBER
I'm not a big fan of having two functions that do the same thing. What do you think ?
2021-12-27T16:27:05Z
https://github.com/huggingface/datasets/pull/3477#issuecomment-1001644544
https://api.github.com/repos/huggingface/datasets/issues/3477
IC_kwDODunzps47s-IA
https://api.github.com/repos/huggingface/datasets/issues/comments/1001644544/reactions
2021-12-27T16:27:05Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1001644544
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
They do not do the same thing:
- One iterates over files in a directory
- The other I guess will iterate over the members of an archive
2021-12-27T16:30:55Z
https://github.com/huggingface/datasets/pull/3477#issuecomment-1001646077
https://api.github.com/repos/huggingface/datasets/issues/3477
IC_kwDODunzps47s-f9
https://api.github.com/repos/huggingface/datasets/issues/comments/1001646077/reactions
2021-12-27T16:30:55Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1001646077
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
MEMBER
Makes sense ! Sounds good then - sorry for my misunderstanding

Note that `iter_archive` will be more performant for data streaming that `iter_files` thanks to the buffering so maybe in the future we can `iter_archive` for some of these datasets
2021-12-27T16:53:41Z
https://github.com/huggingface/datasets/pull/3477#issuecomment-1001654061
https://api.github.com/repos/huggingface/datasets/issues/3477
IC_kwDODunzps47tAct
https://api.github.com/repos/huggingface/datasets/issues/comments/1001654061/reactions
2021-12-27T16:53:41Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1001654061
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
Yes, @lhoestq I agree with you: once `iter_archive` supports zip files, it will be more suitable than `iter_files` for these 2 datasets.

Anyway, this PR also implements `isfile`/`isdir` in streaming mode, besides fixing `iter_files`. And I'm interested in having those in master.

Maybe, could we merge this PR into master and take note to refactor the datasets to use `iter_archive` once zip is supported?
Other option could be to split this PR into 2...
2021-12-28T10:43:48Z
https://github.com/huggingface/datasets/pull/3477#issuecomment-1002006252
https://api.github.com/repos/huggingface/datasets/issues/3477
IC_kwDODunzps47uWbs
https://api.github.com/repos/huggingface/datasets/issues/comments/1002006252/reactions
2021-12-28T10:43:48Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1002006252
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
Use `iter_files` in the `beans` and the `cats_vs_dogs` dataset scripts as suggested by @albertvillanova.

Additional changes:
* Fix `iter_files` in `MockDownloadManager` (see this [CI error](https://app.circleci.com/pipelines/github/huggingface/datasets/9247/workflows/2657ff8a-b531-4fd9-a9fc-6541a72e8d83/jobs/57028))
* Add support for `os.path.isdir` and `os.path.isfile` in streaming (`os.path.isfile` is needed in `StreamingDownloadManager`'s `iter_files` to make `cats_vs_dogs` streamable)

TODO:
- [ ] add tests for `xisdir` and `xisfile`