Drop labels in Image and Audio folders if files are on different levels in directory or if there is only one label
NONE
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_5192). All of your documentation changes will be reflected on that endpoint.
2022-11-02T14:07:47Z
https://github.com/huggingface/datasets/pull/5192#issuecomment-1300480366
https://api.github.com/repos/huggingface/datasets/issues/5192
IC_kwDODunzps5Ng8Fu
https://api.github.com/repos/huggingface/datasets/issues/comments/1300480366/reactions
2022-11-02T14:07:47Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1300480366
https://avatars.githubusercontent.com/u/99929124?v=4
https://api.github.com/users/HuggingFaceDocBuilderDev/events{/privacy}
https://api.github.com/users/HuggingFaceDocBuilderDev/followers
https://api.github.com/users/HuggingFaceDocBuilderDev/following{/other_user}
https://api.github.com/users/HuggingFaceDocBuilderDev/gists{/gist_id}
https://github.com/HuggingFaceDocBuilderDev
HuggingFaceDocBuilderDev
U_kgDOBfTMJA
https://api.github.com/users/HuggingFaceDocBuilderDev/orgs
https://api.github.com/users/HuggingFaceDocBuilderDev/received_events
https://api.github.com/users/HuggingFaceDocBuilderDev/repos
https://api.github.com/users/HuggingFaceDocBuilderDev/starred{/owner}{/repo}
https://api.github.com/users/HuggingFaceDocBuilderDev/subscriptions
User
https://api.github.com/users/HuggingFaceDocBuilderDev
public
NONE
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_5192). All of your documentation changes will be reflected on that endpoint.
2022-11-14T10:43:27Z
https://github.com/huggingface/datasets/pull/5192#issuecomment-1313461360
https://api.github.com/repos/huggingface/datasets/issues/5192
IC_kwDODunzps5OSdRw
https://api.github.com/repos/huggingface/datasets/issues/comments/1313461360/reactions
2022-11-14T10:43:27Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1313461360
https://avatars.githubusercontent.com/u/99929124?v=4
https://api.github.com/users/HuggingFaceDocBuilderDev/events{/privacy}
https://api.github.com/users/HuggingFaceDocBuilderDev/followers
https://api.github.com/users/HuggingFaceDocBuilderDev/following{/other_user}
https://api.github.com/users/HuggingFaceDocBuilderDev/gists{/gist_id}
https://github.com/HuggingFaceDocBuilderDev
HuggingFaceDocBuilderDev
U_kgDOBfTMJA
https://api.github.com/users/HuggingFaceDocBuilderDev/orgs
https://api.github.com/users/HuggingFaceDocBuilderDev/received_events
https://api.github.com/users/HuggingFaceDocBuilderDev/repos
https://api.github.com/users/HuggingFaceDocBuilderDev/starred{/owner}{/repo}
https://api.github.com/users/HuggingFaceDocBuilderDev/subscriptions
User
https://api.github.com/users/HuggingFaceDocBuilderDev
public
NONE
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_5192). All of your documentation changes will be reflected on that endpoint.
2022-11-14T15:46:48Z
https://github.com/huggingface/datasets/pull/5192#issuecomment-1313962573
https://api.github.com/repos/huggingface/datasets/issues/5192
IC_kwDODunzps5OUXpN
https://api.github.com/repos/huggingface/datasets/issues/comments/1313962573/reactions
2022-11-14T15:46:48Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1313962573
https://avatars.githubusercontent.com/u/99929124?v=4
https://api.github.com/users/HuggingFaceDocBuilderDev/events{/privacy}
https://api.github.com/users/HuggingFaceDocBuilderDev/followers
https://api.github.com/users/HuggingFaceDocBuilderDev/following{/other_user}
https://api.github.com/users/HuggingFaceDocBuilderDev/gists{/gist_id}
https://github.com/HuggingFaceDocBuilderDev
HuggingFaceDocBuilderDev
U_kgDOBfTMJA
https://api.github.com/users/HuggingFaceDocBuilderDev/orgs
https://api.github.com/users/HuggingFaceDocBuilderDev/received_events
https://api.github.com/users/HuggingFaceDocBuilderDev/repos
https://api.github.com/users/HuggingFaceDocBuilderDev/starred{/owner}{/repo}
https://api.github.com/users/HuggingFaceDocBuilderDev/subscriptions
User
https://api.github.com/users/HuggingFaceDocBuilderDev
public
NONE
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_5192). All of your documentation changes will be reflected on that endpoint.
2022-11-14T16:03:32Z
https://github.com/huggingface/datasets/pull/5192#issuecomment-1313988376
https://api.github.com/repos/huggingface/datasets/issues/5192
IC_kwDODunzps5OUd8Y
https://api.github.com/repos/huggingface/datasets/issues/comments/1313988376/reactions
2022-11-14T16:03:32Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1313988376
https://avatars.githubusercontent.com/u/99929124?v=4
https://api.github.com/users/HuggingFaceDocBuilderDev/events{/privacy}
https://api.github.com/users/HuggingFaceDocBuilderDev/followers
https://api.github.com/users/HuggingFaceDocBuilderDev/following{/other_user}
https://api.github.com/users/HuggingFaceDocBuilderDev/gists{/gist_id}
https://github.com/HuggingFaceDocBuilderDev
HuggingFaceDocBuilderDev
U_kgDOBfTMJA
https://api.github.com/users/HuggingFaceDocBuilderDev/orgs
https://api.github.com/users/HuggingFaceDocBuilderDev/received_events
https://api.github.com/users/HuggingFaceDocBuilderDev/repos
https://api.github.com/users/HuggingFaceDocBuilderDev/starred{/owner}{/repo}
https://api.github.com/users/HuggingFaceDocBuilderDev/subscriptions
User
https://api.github.com/users/HuggingFaceDocBuilderDev
public
NONE
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_5192). All of your documentation changes will be reflected on that endpoint.
2022-11-15T08:01:05Z
https://github.com/huggingface/datasets/pull/5192#issuecomment-1314927937
https://api.github.com/repos/huggingface/datasets/issues/5192
IC_kwDODunzps5OYDVB
https://api.github.com/repos/huggingface/datasets/issues/comments/1314927937/reactions
2022-11-15T08:01:05Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1314927937
https://avatars.githubusercontent.com/u/99929124?v=4
https://api.github.com/users/HuggingFaceDocBuilderDev/events{/privacy}
https://api.github.com/users/HuggingFaceDocBuilderDev/followers
https://api.github.com/users/HuggingFaceDocBuilderDev/following{/other_user}
https://api.github.com/users/HuggingFaceDocBuilderDev/gists{/gist_id}
https://github.com/HuggingFaceDocBuilderDev
HuggingFaceDocBuilderDev
U_kgDOBfTMJA
https://api.github.com/users/HuggingFaceDocBuilderDev/orgs
https://api.github.com/users/HuggingFaceDocBuilderDev/received_events
https://api.github.com/users/HuggingFaceDocBuilderDev/repos
https://api.github.com/users/HuggingFaceDocBuilderDev/starred{/owner}{/repo}
https://api.github.com/users/HuggingFaceDocBuilderDev/subscriptions
User
https://api.github.com/users/HuggingFaceDocBuilderDev
public
CONTRIBUTOR
> Nit: maybe we can use the count_path_segments function from this file for counting (updated with your logic to make it faster).

@mariosasko just to make sure I understood you correctly - are you okay with this change? (actually `os.path.normpath` is redundant here as paths from `data_files` should be already normalized but just in case)
https://github.com/huggingface/datasets/pull/5192/files#diff-1f09f7a178211f7539b1499b64b69793bd53b30c8b7b34cfcc5835e25d31929fR33
If you are, we can merge.
2022-11-15T08:24:37Z
https://github.com/huggingface/datasets/pull/5192#issuecomment-1314956545
https://api.github.com/repos/huggingface/datasets/issues/5192
IC_kwDODunzps5OYKUB
https://api.github.com/repos/huggingface/datasets/issues/comments/1314956545/reactions
2022-11-15T08:24:37Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1314956545
https://avatars.githubusercontent.com/u/16348744?v=4
https://api.github.com/users/polinaeterna/events{/privacy}
https://api.github.com/users/polinaeterna/followers
https://api.github.com/users/polinaeterna/following{/other_user}
https://api.github.com/users/polinaeterna/gists{/gist_id}
https://github.com/polinaeterna
polinaeterna
MDQ6VXNlcjE2MzQ4NzQ0
https://api.github.com/users/polinaeterna/orgs
https://api.github.com/users/polinaeterna/received_events
https://api.github.com/users/polinaeterna/repos
https://api.github.com/users/polinaeterna/starred{/owner}{/repo}
https://api.github.com/users/polinaeterna/subscriptions
User
https://api.github.com/users/polinaeterna
public
NONE
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_5192). All of your documentation changes will be reflected on that endpoint.
2022-11-15T15:31:08Z
https://github.com/huggingface/datasets/pull/5192#issuecomment-1315478450
https://api.github.com/repos/huggingface/datasets/issues/5192
IC_kwDODunzps5OaJuy
https://api.github.com/repos/huggingface/datasets/issues/comments/1315478450/reactions
2022-11-15T15:31:08Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1315478450
https://avatars.githubusercontent.com/u/99929124?v=4
https://api.github.com/users/HuggingFaceDocBuilderDev/events{/privacy}
https://api.github.com/users/HuggingFaceDocBuilderDev/followers
https://api.github.com/users/HuggingFaceDocBuilderDev/following{/other_user}
https://api.github.com/users/HuggingFaceDocBuilderDev/gists{/gist_id}
https://github.com/HuggingFaceDocBuilderDev
HuggingFaceDocBuilderDev
U_kgDOBfTMJA
https://api.github.com/users/HuggingFaceDocBuilderDev/orgs
https://api.github.com/users/HuggingFaceDocBuilderDev/received_events
https://api.github.com/users/HuggingFaceDocBuilderDev/repos
https://api.github.com/users/HuggingFaceDocBuilderDev/starred{/owner}{/repo}
https://api.github.com/users/HuggingFaceDocBuilderDev/subscriptions
User
https://api.github.com/users/HuggingFaceDocBuilderDev
public
NONE
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_5192). All of your documentation changes will be reflected on that endpoint.
2022-11-15T15:56:21Z
https://github.com/huggingface/datasets/pull/5192#issuecomment-1315514429
https://api.github.com/repos/huggingface/datasets/issues/5192
IC_kwDODunzps5OaSg9
https://api.github.com/repos/huggingface/datasets/issues/comments/1315514429/reactions
2022-11-15T15:56:21Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1315514429
https://avatars.githubusercontent.com/u/99929124?v=4
https://api.github.com/users/HuggingFaceDocBuilderDev/events{/privacy}
https://api.github.com/users/HuggingFaceDocBuilderDev/followers
https://api.github.com/users/HuggingFaceDocBuilderDev/following{/other_user}
https://api.github.com/users/HuggingFaceDocBuilderDev/gists{/gist_id}
https://github.com/HuggingFaceDocBuilderDev
HuggingFaceDocBuilderDev
U_kgDOBfTMJA
https://api.github.com/users/HuggingFaceDocBuilderDev/orgs
https://api.github.com/users/HuggingFaceDocBuilderDev/received_events
https://api.github.com/users/HuggingFaceDocBuilderDev/repos
https://api.github.com/users/HuggingFaceDocBuilderDev/starred{/owner}{/repo}
https://api.github.com/users/HuggingFaceDocBuilderDev/subscriptions
User
https://api.github.com/users/HuggingFaceDocBuilderDev
public
MEMBER
awesome ! :D
2022-11-15T16:32:53Z
https://github.com/huggingface/datasets/pull/5192#issuecomment-1315568659
https://api.github.com/repos/huggingface/datasets/issues/5192
IC_kwDODunzps5OafwT
https://api.github.com/repos/huggingface/datasets/issues/comments/1315568659/reactions
2022-11-15T16:32:53Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1315568659
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
Will close https://github.com/huggingface/datasets/issues/5153

Drop labels by default (`drop_labels=None`) when:
* there are files on different levels of directory hierarchy by checking their path depth
* all files are in the same directory (=only one label was inferred)

First one fixes cases like this:
```
repo
   image3.jpg
   image4.jpg
   data
      image1.jpg
      image2.jpg
```
Second one fixes cases like this:
```
repo
   image1.jpg
   image2.jpg
   image3.jpg
```
This is mostly to fix the viewer for people who just drop images in the Hub interface into the root dir.

I added tests for both of the cases on local and remote files. **I also changed data files for old test on drop_labels** (`test_generate_examples_drop_labels`). The files I provide to `test_generate_examples_drop_labels` now has "canonical" classification structure (two dirs) in order not to change the logic of the test (=not to check these two cases addressed in the PR).

