Automatically add filename for image/audio folder
CONTRIBUTOR
Also cc @anton-l 
2022-10-25T09:56:57Z
https://github.com/huggingface/datasets/issues/5160#issuecomment-1290292427
https://api.github.com/repos/huggingface/datasets/issues/5160
IC_kwDODunzps5M6EzL
https://api.github.com/repos/huggingface/datasets/issues/comments/1290292427/reactions
2022-10-25T09:56:57Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1290292427
https://avatars.githubusercontent.com/u/23423619?v=4
https://api.github.com/users/patrickvonplaten/events{/privacy}
https://api.github.com/users/patrickvonplaten/followers
https://api.github.com/users/patrickvonplaten/following{/other_user}
https://api.github.com/users/patrickvonplaten/gists{/gist_id}
https://github.com/patrickvonplaten
patrickvonplaten
MDQ6VXNlcjIzNDIzNjE5
https://api.github.com/users/patrickvonplaten/orgs
https://api.github.com/users/patrickvonplaten/received_events
https://api.github.com/users/patrickvonplaten/repos
https://api.github.com/users/patrickvonplaten/starred{/owner}{/repo}
https://api.github.com/users/patrickvonplaten/subscriptions
User
https://api.github.com/users/patrickvonplaten
public
CONTRIBUTOR
BTW the exact same holds true for the audio folder
2022-10-25T09:57:09Z
https://github.com/huggingface/datasets/issues/5160#issuecomment-1290292639
https://api.github.com/repos/huggingface/datasets/issues/5160
IC_kwDODunzps5M6E2f
https://api.github.com/repos/huggingface/datasets/issues/comments/1290292639/reactions
2022-10-25T09:57:09Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1290292639
https://avatars.githubusercontent.com/u/23423619?v=4
https://api.github.com/users/patrickvonplaten/events{/privacy}
https://api.github.com/users/patrickvonplaten/followers
https://api.github.com/users/patrickvonplaten/following{/other_user}
https://api.github.com/users/patrickvonplaten/gists{/gist_id}
https://github.com/patrickvonplaten
patrickvonplaten
MDQ6VXNlcjIzNDIzNjE5
https://api.github.com/users/patrickvonplaten/orgs
https://api.github.com/users/patrickvonplaten/received_events
https://api.github.com/users/patrickvonplaten/repos
https://api.github.com/users/patrickvonplaten/starred{/owner}{/repo}
https://api.github.com/users/patrickvonplaten/subscriptions
User
https://api.github.com/users/patrickvonplaten
public
MEMBER
I'm fine with adding a new column with the file name personally. Not sure how breaking this is though
2022-10-25T12:43:46Z
https://github.com/huggingface/datasets/issues/5160#issuecomment-1290495354
https://api.github.com/repos/huggingface/datasets/issues/5160
IC_kwDODunzps5M62V6
https://api.github.com/repos/huggingface/datasets/issues/comments/1290495354/reactions
2022-10-25T12:43:46Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1290495354
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
@patrickvonplaten do you mean just filename or full relative path inside the repo?
I think it shouldn't be breaking, at least I cannot come up with any case where it is. Maybe @mariosasko can?

also I think that the problem here and in general is that Image/AudioFolder has default configuration which implies automatic label creation if there is not metadata file. It can be changed when you load the dataset with `load_dataset` but not on it's Hub page. 


2022-10-25T12:50:03Z
https://github.com/huggingface/datasets/issues/5160#issuecomment-1290503607
https://api.github.com/repos/huggingface/datasets/issues/5160
IC_kwDODunzps5M64W3
https://api.github.com/repos/huggingface/datasets/issues/comments/1290503607/reactions
2022-10-25T15:59:16Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1290503607
https://avatars.githubusercontent.com/u/16348744?v=4
https://api.github.com/users/polinaeterna/events{/privacy}
https://api.github.com/users/polinaeterna/followers
https://api.github.com/users/polinaeterna/following{/other_user}
https://api.github.com/users/polinaeterna/gists{/gist_id}
https://github.com/polinaeterna
polinaeterna
MDQ6VXNlcjE2MzQ4NzQ0
https://api.github.com/users/polinaeterna/orgs
https://api.github.com/users/polinaeterna/received_events
https://api.github.com/users/polinaeterna/repos
https://api.github.com/users/polinaeterna/starred{/owner}{/repo}
https://api.github.com/users/polinaeterna/subscriptions
User
https://api.github.com/users/polinaeterna
public
MEMBER
> also I think that the problem here and in general Image/AudioFolder has default configuration which implies automatic label creation if there is not metadata file

Yea I agree it's often the wrong default. We can also imagine adding the builder's parameters as YAML in the repo.
2022-10-25T12:53:56Z
https://github.com/huggingface/datasets/issues/5160#issuecomment-1290508173
https://api.github.com/repos/huggingface/datasets/issues/5160
IC_kwDODunzps5M65eN
https://api.github.com/repos/huggingface/datasets/issues/comments/1290508173/reactions
2022-10-25T12:53:56Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1290508173
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
@lhoestq yes I also got the idea of some YAML config! not sure of what priority it is though.
2022-10-25T12:56:22Z
https://github.com/huggingface/datasets/issues/5160#issuecomment-1290511102
https://api.github.com/repos/huggingface/datasets/issues/5160
IC_kwDODunzps5M66L-
https://api.github.com/repos/huggingface/datasets/issues/comments/1290511102/reactions
2022-10-25T12:56:22Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1290511102
https://avatars.githubusercontent.com/u/16348744?v=4
https://api.github.com/users/polinaeterna/events{/privacy}
https://api.github.com/users/polinaeterna/followers
https://api.github.com/users/polinaeterna/following{/other_user}
https://api.github.com/users/polinaeterna/gists{/gist_id}
https://github.com/polinaeterna
polinaeterna
MDQ6VXNlcjE2MzQ4NzQ0
https://api.github.com/users/polinaeterna/orgs
https://api.github.com/users/polinaeterna/received_events
https://api.github.com/users/polinaeterna/repos
https://api.github.com/users/polinaeterna/starred{/owner}{/repo}
https://api.github.com/users/polinaeterna/subscriptions
User
https://api.github.com/users/polinaeterna
public
CONTRIBUTOR
but it would actually also solve this issue: https://github.com/huggingface/datasets/issues/5153
2022-10-25T13:01:38Z
https://github.com/huggingface/datasets/issues/5160#issuecomment-1290517374
https://api.github.com/repos/huggingface/datasets/issues/5160
IC_kwDODunzps5M67t-
https://api.github.com/repos/huggingface/datasets/issues/comments/1290517374/reactions
2022-10-25T13:01:38Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1290517374
https://avatars.githubusercontent.com/u/16348744?v=4
https://api.github.com/users/polinaeterna/events{/privacy}
https://api.github.com/users/polinaeterna/followers
https://api.github.com/users/polinaeterna/following{/other_user}
https://api.github.com/users/polinaeterna/gists{/gist_id}
https://github.com/polinaeterna
polinaeterna
MDQ6VXNlcjE2MzQ4NzQ0
https://api.github.com/users/polinaeterna/orgs
https://api.github.com/users/polinaeterna/received_events
https://api.github.com/users/polinaeterna/repos
https://api.github.com/users/polinaeterna/starred{/owner}{/repo}
https://api.github.com/users/polinaeterna/subscriptions
User
https://api.github.com/users/polinaeterna
public
CONTRIBUTOR
I meant just the file name (no path) that would already be super helpful IMO :-) (maybe dir+filename if there are dirs in the folder)
2022-10-25T17:26:15Z
https://github.com/huggingface/datasets/issues/5160#issuecomment-1290897445
https://api.github.com/repos/huggingface/datasets/issues/5160
IC_kwDODunzps5M8Ygl
https://api.github.com/repos/huggingface/datasets/issues/comments/1290897445/reactions
2022-10-25T17:26:15Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1290897445
https://avatars.githubusercontent.com/u/23423619?v=4
https://api.github.com/users/patrickvonplaten/events{/privacy}
https://api.github.com/users/patrickvonplaten/followers
https://api.github.com/users/patrickvonplaten/following{/other_user}
https://api.github.com/users/patrickvonplaten/gists{/gist_id}
https://github.com/patrickvonplaten
patrickvonplaten
MDQ6VXNlcjIzNDIzNjE5
https://api.github.com/users/patrickvonplaten/orgs
https://api.github.com/users/patrickvonplaten/received_events
https://api.github.com/users/patrickvonplaten/repos
https://api.github.com/users/patrickvonplaten/starred{/owner}{/repo}
https://api.github.com/users/patrickvonplaten/subscriptions
User
https://api.github.com/users/patrickvonplaten
public
CONTRIBUTOR
@patrickvonplaten one more time, to be sure I understand you.
For example, we have data structure like this:
```
├─ data/
│  └─ subdir/
│     └── cats/
│          ├── 0.jpg
│          ├── 1.jpg
│          └── 2.jpg
│     └── dogs/
│          ├── 0.jpg
│          ├── 1.jpg
│          └── 2.jpg
└── another_subdir/
      ├── 10.jpg
      ├── 11.jpg
      └── 12.jpg
```
Is it okay to provide `"data/subdir/cats/0.jpg"`, `"data/subdir/dogs/0.jpg"`, `"data/another_subdir/10.jpg"`?
I think providing just filenames might be confusing if they are not unique, as in this example. 
2022-10-26T15:26:17Z
https://github.com/huggingface/datasets/issues/5160#issuecomment-1292222157
https://api.github.com/repos/huggingface/datasets/issues/5160
IC_kwDODunzps5NBb7N
https://api.github.com/repos/huggingface/datasets/issues/comments/1292222157/reactions
2022-10-26T15:26:17Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1292222157
https://avatars.githubusercontent.com/u/16348744?v=4
https://api.github.com/users/polinaeterna/events{/privacy}
https://api.github.com/users/polinaeterna/followers
https://api.github.com/users/polinaeterna/following{/other_user}
https://api.github.com/users/polinaeterna/gists{/gist_id}
https://github.com/polinaeterna
polinaeterna
MDQ6VXNlcjE2MzQ4NzQ0
https://api.github.com/users/polinaeterna/orgs
https://api.github.com/users/polinaeterna/received_events
https://api.github.com/users/polinaeterna/repos
https://api.github.com/users/polinaeterna/starred{/owner}{/repo}
https://api.github.com/users/polinaeterna/subscriptions
User
https://api.github.com/users/polinaeterna
public
CONTRIBUTOR
Yes I think the relative path as you proposed makes a lot of sense :-) 
2022-10-26T16:51:45Z
https://github.com/huggingface/datasets/issues/5160#issuecomment-1292330324
https://api.github.com/repos/huggingface/datasets/issues/5160
IC_kwDODunzps5NB2VU
https://api.github.com/repos/huggingface/datasets/issues/comments/1292330324/reactions
2022-10-26T16:51:45Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1292330324
https://avatars.githubusercontent.com/u/23423619?v=4
https://api.github.com/users/patrickvonplaten/events{/privacy}
https://api.github.com/users/patrickvonplaten/followers
https://api.github.com/users/patrickvonplaten/following{/other_user}
https://api.github.com/users/patrickvonplaten/gists{/gist_id}
https://github.com/patrickvonplaten
patrickvonplaten
MDQ6VXNlcjIzNDIzNjE5
https://api.github.com/users/patrickvonplaten/orgs
https://api.github.com/users/patrickvonplaten/received_events
https://api.github.com/users/patrickvonplaten/repos
https://api.github.com/users/patrickvonplaten/starred{/owner}{/repo}
https://api.github.com/users/patrickvonplaten/subscriptions
User
https://api.github.com/users/patrickvonplaten
public
### Feature request

When creating a custom audio of image dataset, it would be great to automatically have access to the filename. It should be both:
a) Automatically displayed in the viewer
b) Automatically added as a column to the dataset when doing `load_dataset`

In `diffusers` our test rely quite heavily on images and audio files now and it's a bit tedious at the moment to download specific images from a datasets repo. 

E.g. we have a dataset of images for tests in `diffusers`: https://huggingface.co/datasets/hf-internal-testing/diffusers-images
where it would be extremely nice to have direct access to the filename both visually on the datasets page (@severo ) as well as via the `load_datasets` function. We currently have some akward functionality to download images by path name: https://github.com/huggingface/diffusers/blob/2fb8fafa4b761f6fc144cf75a6f6f0ea6af3a1c1/src/diffusers/utils/testing_utils.py#L131 
It would be much nicer to just go over `load_dataset(...)` 

### Motivation

Intuitively the filename is something people understand directly. E.g if you upload a folder of images online, it's nice if you recognize the image as well as the filename next to it directly and that you're able to use it right away. 

The label on the other hand is less intuitive to understand as you haven't added it yourself.

### Your contribution

Not sure if I have the time to add it myself anytime soon, but it would help us a lot for `diffusers`.