Multi-image loading in Imagefolder dataset
COLLABORATOR
Supporting this could be useful (I remember a use-case for this on the Hub). Do you agree @polinaeterna? 

Implementing this should be possible if we iterate over metadata files and build image/audio file paths instead of iterating over image/audio files and looking for the corresponding entries in metadata files.
2023-05-05T15:43:10Z
https://github.com/huggingface/datasets/issues/5760#issuecomment-1536444288
https://api.github.com/repos/huggingface/datasets/issues/5760
IC_kwDODunzps5blEeA
https://api.github.com/repos/huggingface/datasets/issues/comments/1536444288/reactions
2023-05-05T15:43:10Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1536444288
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
NONE
I've build a similar feature from scratch and would be interested to combine it with the datasets package.

My solution works something like this:
Interpret the first element of each column as a file path. If the path exists and is a file, (try to) load the files for the entire column. Thereby, one isn't restricted to a particular column name, with comes in handy when dealing with multiple file columns.

I've looked into the code to try to implement this, but didn't find the right places. I'm also open to contribute, but will need some guidance.
2023-05-16T10:14:59Z
https://github.com/huggingface/datasets/issues/5760#issuecomment-1549386474
https://api.github.com/repos/huggingface/datasets/issues/5760
IC_kwDODunzps5cWcLq
https://api.github.com/repos/huggingface/datasets/issues/comments/1549386474/reactions
2023-05-16T10:14:59Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1549386474
https://avatars.githubusercontent.com/u/56480601?v=4
https://api.github.com/users/dennis-rall/events{/privacy}
https://api.github.com/users/dennis-rall/followers
https://api.github.com/users/dennis-rall/following{/other_user}
https://api.github.com/users/dennis-rall/gists{/gist_id}
https://github.com/dennis-rall
dennis-rall
MDQ6VXNlcjU2NDgwNjAx
https://api.github.com/users/dennis-rall/orgs
https://api.github.com/users/dennis-rall/received_events
https://api.github.com/users/dennis-rall/repos
https://api.github.com/users/dennis-rall/starred{/owner}{/repo}
https://api.github.com/users/dennis-rall/subscriptions
User
https://api.github.com/users/dennis-rall
public
COLLABORATOR
Required here: https://discuss.huggingface.co/t/dataset-repo-requires-arbitrary-python-code-execution/59346/14
2023-11-30T12:06:19Z
https://github.com/huggingface/datasets/issues/5760#issuecomment-1833640843
https://api.github.com/repos/huggingface/datasets/issues/5760
IC_kwDODunzps5tSyOL
https://api.github.com/repos/huggingface/datasets/issues/comments/1833640843/reactions
2023-11-30T12:06:19Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1833640843
https://avatars.githubusercontent.com/u/1676121?v=4
https://api.github.com/users/severo/events{/privacy}
https://api.github.com/users/severo/followers
https://api.github.com/users/severo/following{/other_user}
https://api.github.com/users/severo/gists{/gist_id}
https://github.com/severo
severo
MDQ6VXNlcjE2NzYxMjE=
https://api.github.com/users/severo/orgs
https://api.github.com/users/severo/received_events
https://api.github.com/users/severo/repos
https://api.github.com/users/severo/starred{/owner}{/repo}
https://api.github.com/users/severo/subscriptions
User
https://api.github.com/users/severo
public
NONE
+1

Is the only way to do this right now to write a custom dataset loader script?
2024-03-23T21:50:37Z
https://github.com/huggingface/datasets/issues/5760#issuecomment-2016611900
https://api.github.com/repos/huggingface/datasets/issues/5760
IC_kwDODunzps54Mw48
https://api.github.com/repos/huggingface/datasets/issues/comments/2016611900/reactions
2024-03-23T21:50:37Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2016611900
https://avatars.githubusercontent.com/u/177505?v=4
https://api.github.com/users/whydna/events{/privacy}
https://api.github.com/users/whydna/followers
https://api.github.com/users/whydna/following{/other_user}
https://api.github.com/users/whydna/gists{/gist_id}
https://github.com/whydna
whydna
MDQ6VXNlcjE3NzUwNQ==
https://api.github.com/users/whydna/orgs
https://api.github.com/users/whydna/received_events
https://api.github.com/users/whydna/repos
https://api.github.com/users/whydna/starred{/owner}{/repo}
https://api.github.com/users/whydna/subscriptions
User
https://api.github.com/users/whydna
public
COLLABORATOR
Also: be able to have input and output images for each row. Asked here: https://discuss.huggingface.co/t/how-to-structure-image-files-for-datasets-load-dataset-imagefolder-when-you-have-input-and-output-images-like-in-instruct-pix2pix/82467
2024-04-19T07:48:38Z
https://github.com/huggingface/datasets/issues/5760#issuecomment-2065994529
https://api.github.com/repos/huggingface/datasets/issues/5760
IC_kwDODunzps57JJMh
https://api.github.com/repos/huggingface/datasets/issues/comments/2065994529/reactions
2024-04-19T07:48:38Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2065994529
https://avatars.githubusercontent.com/u/1676121?v=4
https://api.github.com/users/severo/events{/privacy}
https://api.github.com/users/severo/followers
https://api.github.com/users/severo/following{/other_user}
https://api.github.com/users/severo/gists{/gist_id}
https://github.com/severo
severo
MDQ6VXNlcjE2NzYxMjE=
https://api.github.com/users/severo/orgs
https://api.github.com/users/severo/received_events
https://api.github.com/users/severo/repos
https://api.github.com/users/severo/starred{/owner}{/repo}
https://api.github.com/users/severo/subscriptions
User
https://api.github.com/users/severo
public
NONE
👀 I encountered the same problem. Is the only way to solve it by writing a custom dataset loader script?
2024-12-01T10:51:59Z
https://github.com/huggingface/datasets/issues/5760#issuecomment-2509684773
https://api.github.com/repos/huggingface/datasets/issues/5760
IC_kwDODunzps6VlsAl
https://api.github.com/repos/huggingface/datasets/issues/comments/2509684773/reactions
2024-12-01T10:51:59Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2509684773
https://avatars.githubusercontent.com/u/91264713?v=4
https://api.github.com/users/hychaochao/events{/privacy}
https://api.github.com/users/hychaochao/followers
https://api.github.com/users/hychaochao/following{/other_user}
https://api.github.com/users/hychaochao/gists{/gist_id}
https://github.com/hychaochao
hychaochao
MDQ6VXNlcjkxMjY0NzEz
https://api.github.com/users/hychaochao/orgs
https://api.github.com/users/hychaochao/received_events
https://api.github.com/users/hychaochao/repos
https://api.github.com/users/hychaochao/starred{/owner}{/repo}
https://api.github.com/users/hychaochao/subscriptions
User
https://api.github.com/users/hychaochao
public
NONE
Yes I had to use script at the end

Sent from Outlook for iOS<https://aka.ms/o0ukef>
________________________________
From: Yunzhuo Hao ***@***.***>
Sent: Sunday, December 1, 2024 11:52:21 AM
To: huggingface/datasets ***@***.***>
Cc: Sushant Gautam ***@***.***>; Manual ***@***.***>
Subject: Re: [huggingface/datasets] Multi-image loading in Imagefolder dataset (Issue #5760)


👀 I encountered the same problem. Is the only way to solve it by writing a custom dataset loader script?

—
Reply to this email directly, view it on GitHub<https://github.com/huggingface/datasets/issues/5760#issuecomment-2509684773>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AD7SQP5KY67NTZ7ELAGSIJ32DLS6LAVCNFSM6AAAAABSZRPAQOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMBZGY4DINZXGM>.
You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

2024-12-01T10:54:04Z
https://github.com/huggingface/datasets/issues/5760#issuecomment-2509685427
https://api.github.com/repos/huggingface/datasets/issues/5760
IC_kwDODunzps6VlsKz
https://api.github.com/repos/huggingface/datasets/issues/comments/2509685427/reactions
2024-12-01T10:54:04Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2509685427
https://avatars.githubusercontent.com/u/16721983?v=4
https://api.github.com/users/SushantGautam/events{/privacy}
https://api.github.com/users/SushantGautam/followers
https://api.github.com/users/SushantGautam/following{/other_user}
https://api.github.com/users/SushantGautam/gists{/gist_id}
https://github.com/SushantGautam
SushantGautam
MDQ6VXNlcjE2NzIxOTgz
https://api.github.com/users/SushantGautam/orgs
https://api.github.com/users/SushantGautam/received_events
https://api.github.com/users/SushantGautam/repos
https://api.github.com/users/SushantGautam/starred{/owner}{/repo}
https://api.github.com/users/SushantGautam/subscriptions
User
https://api.github.com/users/SushantGautam
public
NONE
@SushantGautam Thanks for your reply!! In addition, can the dataset shared by writing such a script be displayed in the Viewer? And if it is convenient, can you please show me your script? Thanks again!!
2024-12-01T11:02:19Z
https://github.com/huggingface/datasets/issues/5760#issuecomment-2509692544
https://api.github.com/repos/huggingface/datasets/issues/5760
IC_kwDODunzps6Vlt6A
https://api.github.com/repos/huggingface/datasets/issues/comments/2509692544/reactions
2024-12-01T11:02:19Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2509692544
https://avatars.githubusercontent.com/u/91264713?v=4
https://api.github.com/users/hychaochao/events{/privacy}
https://api.github.com/users/hychaochao/followers
https://api.github.com/users/hychaochao/following{/other_user}
https://api.github.com/users/hychaochao/gists{/gist_id}
https://github.com/hychaochao
hychaochao
MDQ6VXNlcjkxMjY0NzEz
https://api.github.com/users/hychaochao/orgs
https://api.github.com/users/hychaochao/received_events
https://api.github.com/users/hychaochao/repos
https://api.github.com/users/hychaochao/starred{/owner}{/repo}
https://api.github.com/users/hychaochao/subscriptions
User
https://api.github.com/users/hychaochao
public
NONE
Yes will be visible!
The script docs is at
https://huggingface.co/docs/datasets/en/dataset_script



Sincerely,
*Sushant Gautam*

📧: ***@***.*** | ***@***.***
🌐: sushant.info.np | simula.no/people/sushant
<https://www.simula.no/people/sushant>
[image: Simula Metropolitan Center for Digital Engineering]
<https://www.simulamet.no/>
Stensberggata 27, 0170 Oslo
Find me on:* LinkedIn <https://www.linkedin.com/in/eSushant> | Facebook
<https://www.facebook.com/eSushant> | Twitter
<https://twitter.com/esushant> *


On Sun, 1 Dec 2024 at 12:02, Yunzhuo Hao ***@***.***> wrote:

> @SushantGautam <https://github.com/SushantGautam> Thanks for your reply!!
> In addition, can the dataset shared by writing such a script be displayed
> in the Viewer? And if it is convenient, can you please show me your script?
> Thanks again!!
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/datasets/issues/5760#issuecomment-2509692544>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AD7SQP2XGIKBCSSSNW2S6K32DLUFDAVCNFSM6AAAAABSZRPAQOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMBZGY4TENJUGQ>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>

2024-12-01T11:12:27Z
https://github.com/huggingface/datasets/issues/5760#issuecomment-2509717501
https://api.github.com/repos/huggingface/datasets/issues/5760
IC_kwDODunzps6Vlz_9
https://api.github.com/repos/huggingface/datasets/issues/comments/2509717501/reactions
2024-12-01T11:12:27Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2509717501
https://avatars.githubusercontent.com/u/16721983?v=4
https://api.github.com/users/SushantGautam/events{/privacy}
https://api.github.com/users/SushantGautam/followers
https://api.github.com/users/SushantGautam/following{/other_user}
https://api.github.com/users/SushantGautam/gists{/gist_id}
https://github.com/SushantGautam
SushantGautam
MDQ6VXNlcjE2NzIxOTgz
https://api.github.com/users/SushantGautam/orgs
https://api.github.com/users/SushantGautam/received_events
https://api.github.com/users/SushantGautam/repos
https://api.github.com/users/SushantGautam/starred{/owner}{/repo}
https://api.github.com/users/SushantGautam/subscriptions
User
https://api.github.com/users/SushantGautam
public
NONE
I see. Thank you very very much!!!

Sushant Gautam ***@***.***>于2024年12月1日 周日18:54写道：

> Yes I had to use script at the end
>
> Sent from Outlook for iOS<https://aka.ms/o0ukef>
> ________________________________
> From: Yunzhuo Hao ***@***.***>
> Sent: Sunday, December 1, 2024 11:52:21 AM
> To: huggingface/datasets ***@***.***>
> Cc: Sushant Gautam ***@***.***>; Manual ***@***.***>
> Subject: Re: [huggingface/datasets] Multi-image loading in Imagefolder
> dataset (Issue #5760)
>
>
> 👀 I encountered the same problem. Is the only way to solve it by writing
> a custom dataset loader script?
>
> —
> Reply to this email directly, view it on GitHub<
> https://github.com/huggingface/datasets/issues/5760#issuecomment-2509684773>,
> or unsubscribe<
> https://github.com/notifications/unsubscribe-auth/AD7SQP5KY67NTZ7ELAGSIJ32DLS6LAVCNFSM6AAAAABSZRPAQOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMBZGY4DINZXGM>.
>
> You are receiving this because you are subscribed to this thread.Message
> ID: ***@***.***>
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/datasets/issues/5760#issuecomment-2509685427>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AVYJNSJ6UQZ6G6VGSB5RGXD2DLTGFAVCNFSM6AAAAABSZRPAQOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMBZGY4DKNBSG4>
> .
> You are receiving this because you commented.Message ID:
> ***@***.***>
>

2024-12-01T11:16:08Z
https://github.com/huggingface/datasets/issues/5760#issuecomment-2509718764
https://api.github.com/repos/huggingface/datasets/issues/5760
IC_kwDODunzps6Vl0Ts
https://api.github.com/repos/huggingface/datasets/issues/comments/2509718764/reactions
2024-12-01T11:16:08Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2509718764
https://avatars.githubusercontent.com/u/91264713?v=4
https://api.github.com/users/hychaochao/events{/privacy}
https://api.github.com/users/hychaochao/followers
https://api.github.com/users/hychaochao/following{/other_user}
https://api.github.com/users/hychaochao/gists{/gist_id}
https://github.com/hychaochao
hychaochao
MDQ6VXNlcjkxMjY0NzEz
https://api.github.com/users/hychaochao/orgs
https://api.github.com/users/hychaochao/received_events
https://api.github.com/users/hychaochao/repos
https://api.github.com/users/hychaochao/starred{/owner}{/repo}
https://api.github.com/users/hychaochao/subscriptions
User
https://api.github.com/users/hychaochao
public
### Feature request

Extend the `imagefolder` dataloading script to support loading multiple images per dataset entry.

This only really makes sense if a metadata file is present.

Currently you can use the following format (example `metadata.jsonl`:
```
{'file_name': 'path_to_image.png', 'metadata': ...}
...
```
which will return a batch with key `image` and any other metadata.

I would propose extending `file_name` to also accept a list of files, which would return a batch with key `images` and any other metadata.

### Motivation

This is useful for example in segmentation tasks in computer vision models, or in text-to-image models that also accept conditioning signals such as another image, feature map, or similar. Currently if I want to do this, I would need to write a custom dataset, rather than just use `imagefolder`.

### Your contribution

Would be open to doing a PR, but also happy for someone else to take it as I am not familiar with the datasets library.