FolderBase Dataset automatically resolves under current directory when data_dir is not specified
CONTRIBUTOR
@lhoestq 
2023-08-16T04:39:59Z
https://github.com/huggingface/datasets/issues/6152#issuecomment-1679949406
https://api.github.com/repos/huggingface/datasets/issues/6152
IC_kwDODunzps5kIf5e
https://api.github.com/repos/huggingface/datasets/issues/comments/1679949406/reactions
2023-08-16T04:39:59Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1679949406
https://avatars.githubusercontent.com/u/11533479?v=4
https://api.github.com/users/npuichigo/events{/privacy}
https://api.github.com/users/npuichigo/followers
https://api.github.com/users/npuichigo/following{/other_user}
https://api.github.com/users/npuichigo/gists{/gist_id}
https://github.com/npuichigo
npuichigo
MDQ6VXNlcjExNTMzNDc5
https://api.github.com/users/npuichigo/orgs
https://api.github.com/users/npuichigo/received_events
https://api.github.com/users/npuichigo/repos
https://api.github.com/users/npuichigo/starred{/owner}{/repo}
https://api.github.com/users/npuichigo/subscriptions
User
https://api.github.com/users/npuichigo
public
MEMBER
Makes sense, I guess this can be fixed in the load_dataset_builder method.
It concerns every packaged builder I think (see values in `_PACKAGED_DATASETS_MODULES`)
2023-08-16T09:18:01Z
https://github.com/huggingface/datasets/issues/6152#issuecomment-1680255253
https://api.github.com/repos/huggingface/datasets/issues/6152
IC_kwDODunzps5kJqkV
https://api.github.com/repos/huggingface/datasets/issues/comments/1680255253/reactions
2023-08-16T09:18:01Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1680255253
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
I think the behavior is related to these lines, which short circuited the error handling.
https://github.com/huggingface/datasets/blob/664a1cb72ea1e6ef7c47e671e2686ca4a35e8d63/src/datasets/load.py#L946-L952

So should data_dir be checked here or still delegating to actual `DatasetModule`? In that case, how to properly set `data_files` here.
2023-08-16T13:38:11Z
https://github.com/huggingface/datasets/issues/6152#issuecomment-1680623798
https://api.github.com/repos/huggingface/datasets/issues/6152
IC_kwDODunzps5kLEi2
https://api.github.com/repos/huggingface/datasets/issues/comments/1680623798/reactions
2023-08-16T13:38:11Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1680623798
https://avatars.githubusercontent.com/u/11533479?v=4
https://api.github.com/users/npuichigo/events{/privacy}
https://api.github.com/users/npuichigo/followers
https://api.github.com/users/npuichigo/following{/other_user}
https://api.github.com/users/npuichigo/gists{/gist_id}
https://github.com/npuichigo
npuichigo
MDQ6VXNlcjExNTMzNDc5
https://api.github.com/users/npuichigo/orgs
https://api.github.com/users/npuichigo/received_events
https://api.github.com/users/npuichigo/repos
https://api.github.com/users/npuichigo/starred{/owner}{/repo}
https://api.github.com/users/npuichigo/subscriptions
User
https://api.github.com/users/npuichigo
public
MEMBER
This is location in PackagedDatasetModuleFactory.get_module seems the be the right place to check if at least data_dir or data_files are passed
2023-08-16T13:44:29Z
https://github.com/huggingface/datasets/issues/6152#issuecomment-1680639168
https://api.github.com/repos/huggingface/datasets/issues/6152
IC_kwDODunzps5kLITA
https://api.github.com/repos/huggingface/datasets/issues/comments/1680639168/reactions
2023-08-16T13:44:29Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1680639168
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
@mariosasko can you please assign this issue to me,I want to work on this
2023-10-01T04:55:41Z
https://github.com/huggingface/datasets/issues/6152#issuecomment-1741958570
https://api.github.com/repos/huggingface/datasets/issues/6152
IC_kwDODunzps5n1C2q
https://api.github.com/repos/huggingface/datasets/issues/comments/1741958570/reactions
2023-10-01T04:55:41Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1741958570
https://avatars.githubusercontent.com/u/126772439?v=4
https://api.github.com/users/debrupf2946/events{/privacy}
https://api.github.com/users/debrupf2946/followers
https://api.github.com/users/debrupf2946/following{/other_user}
https://api.github.com/users/debrupf2946/gists{/gist_id}
https://github.com/debrupf2946
debrupf2946
U_kgDOB45k1w
https://api.github.com/users/debrupf2946/orgs
https://api.github.com/users/debrupf2946/received_events
https://api.github.com/users/debrupf2946/repos
https://api.github.com/users/debrupf2946/starred{/owner}{/repo}
https://api.github.com/users/debrupf2946/subscriptions
User
https://api.github.com/users/debrupf2946
public
NONE
#self-assign
2023-10-01T05:37:25Z
https://github.com/huggingface/datasets/issues/6152#issuecomment-1741966306
https://api.github.com/repos/huggingface/datasets/issues/6152
IC_kwDODunzps5n1Evi
https://api.github.com/repos/huggingface/datasets/issues/comments/1741966306/reactions
2023-10-01T05:37:25Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1741966306
https://avatars.githubusercontent.com/u/126772439?v=4
https://api.github.com/users/debrupf2946/events{/privacy}
https://api.github.com/users/debrupf2946/followers
https://api.github.com/users/debrupf2946/following{/other_user}
https://api.github.com/users/debrupf2946/gists{/gist_id}
https://github.com/debrupf2946
debrupf2946
U_kgDOB45k1w
https://api.github.com/users/debrupf2946/orgs
https://api.github.com/users/debrupf2946/received_events
https://api.github.com/users/debrupf2946/repos
https://api.github.com/users/debrupf2946/starred{/owner}{/repo}
https://api.github.com/users/debrupf2946/subscriptions
User
https://api.github.com/users/debrupf2946
public
NONE
@mariosasko is this issue still open? i would love to kickstart my journey to open source with this issue!
Regards
zutarich
2023-10-09T06:56:05Z
https://github.com/huggingface/datasets/issues/6152#issuecomment-1752437531
https://api.github.com/repos/huggingface/datasets/issues/6152
IC_kwDODunzps5odBMb
https://api.github.com/repos/huggingface/datasets/issues/comments/1752437531/reactions
2023-10-09T06:56:05Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1752437531
https://avatars.githubusercontent.com/u/95092167?v=4
https://api.github.com/users/zutarich/events{/privacy}
https://api.github.com/users/zutarich/followers
https://api.github.com/users/zutarich/following{/other_user}
https://api.github.com/users/zutarich/gists{/gist_id}
https://github.com/zutarich
zutarich
U_kgDOBar9xw
https://api.github.com/users/zutarich/orgs
https://api.github.com/users/zutarich/received_events
https://api.github.com/users/zutarich/repos
https://api.github.com/users/zutarich/starred{/owner}{/repo}
https://api.github.com/users/zutarich/subscriptions
User
https://api.github.com/users/zutarich
public
COLLABORATOR
@zutarich It is unless @debrupf2946 is working on it.
2023-10-10T16:30:11Z
https://github.com/huggingface/datasets/issues/6152#issuecomment-1755799788
https://api.github.com/repos/huggingface/datasets/issues/6152
IC_kwDODunzps5op2Ds
https://api.github.com/repos/huggingface/datasets/issues/comments/1755799788/reactions
2023-10-10T16:30:19Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1755799788
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
NONE
#self-assign
2024-01-22T14:41:53Z
https://github.com/huggingface/datasets/issues/6152#issuecomment-1904147587
https://api.github.com/repos/huggingface/datasets/issues/6152
IC_kwDODunzps5xfvyD
https://api.github.com/repos/huggingface/datasets/issues/comments/1904147587/reactions
2024-01-22T14:41:53Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1904147587
https://avatars.githubusercontent.com/u/92247226?v=4
https://api.github.com/users/Etelis/events{/privacy}
https://api.github.com/users/Etelis/followers
https://api.github.com/users/Etelis/following{/other_user}
https://api.github.com/users/Etelis/gists{/gist_id}
https://github.com/Etelis
Etelis
U_kgDOBX-Uug
https://api.github.com/users/Etelis/orgs
https://api.github.com/users/Etelis/received_events
https://api.github.com/users/Etelis/repos
https://api.github.com/users/Etelis/starred{/owner}{/repo}
https://api.github.com/users/Etelis/subscriptions
User
https://api.github.com/users/Etelis
public
NONE
I am working and will open a pull request soon @Etelis 

2024-01-22T15:04:51Z
https://github.com/huggingface/datasets/issues/6152#issuecomment-1904195809
https://api.github.com/repos/huggingface/datasets/issues/6152
IC_kwDODunzps5xf7jh
https://api.github.com/repos/huggingface/datasets/issues/comments/1904195809/reactions
2024-01-22T15:04:51Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1904195809
https://avatars.githubusercontent.com/u/126772439?v=4
https://api.github.com/users/debrupf2946/events{/privacy}
https://api.github.com/users/debrupf2946/followers
https://api.github.com/users/debrupf2946/following{/other_user}
https://api.github.com/users/debrupf2946/gists{/gist_id}
https://github.com/debrupf2946
debrupf2946
U_kgDOB45k1w
https://api.github.com/users/debrupf2946/orgs
https://api.github.com/users/debrupf2946/received_events
https://api.github.com/users/debrupf2946/repos
https://api.github.com/users/debrupf2946/starred{/owner}{/repo}
https://api.github.com/users/debrupf2946/subscriptions
User
https://api.github.com/users/debrupf2946
public
CONTRIBUTOR
@mariosasko can i take this up? 
2024-04-03T17:58:02Z
https://github.com/huggingface/datasets/issues/6152#issuecomment-2035253350
https://api.github.com/repos/huggingface/datasets/issues/6152
IC_kwDODunzps55T4Bm
https://api.github.com/repos/huggingface/datasets/issues/comments/2035253350/reactions
2024-04-03T17:58:02Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2035253350
https://avatars.githubusercontent.com/u/63234112?v=4
https://api.github.com/users/JINO-ROHIT/events{/privacy}
https://api.github.com/users/JINO-ROHIT/followers
https://api.github.com/users/JINO-ROHIT/following{/other_user}
https://api.github.com/users/JINO-ROHIT/gists{/gist_id}
https://github.com/JINO-ROHIT
JINO-ROHIT
MDQ6VXNlcjYzMjM0MTEy
https://api.github.com/users/JINO-ROHIT/orgs
https://api.github.com/users/JINO-ROHIT/received_events
https://api.github.com/users/JINO-ROHIT/repos
https://api.github.com/users/JINO-ROHIT/starred{/owner}{/repo}
https://api.github.com/users/JINO-ROHIT/subscriptions
User
https://api.github.com/users/JINO-ROHIT
public
CONTRIBUTOR
#self-assign
2024-04-03T17:58:09Z
https://github.com/huggingface/datasets/issues/6152#issuecomment-2035253533
https://api.github.com/repos/huggingface/datasets/issues/6152
IC_kwDODunzps55T4Ed
https://api.github.com/repos/huggingface/datasets/issues/comments/2035253533/reactions
2024-04-03T17:58:18Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2035253533
https://avatars.githubusercontent.com/u/63234112?v=4
https://api.github.com/users/JINO-ROHIT/events{/privacy}
https://api.github.com/users/JINO-ROHIT/followers
https://api.github.com/users/JINO-ROHIT/following{/other_user}
https://api.github.com/users/JINO-ROHIT/gists{/gist_id}
https://github.com/JINO-ROHIT
JINO-ROHIT
MDQ6VXNlcjYzMjM0MTEy
https://api.github.com/users/JINO-ROHIT/orgs
https://api.github.com/users/JINO-ROHIT/received_events
https://api.github.com/users/JINO-ROHIT/repos
https://api.github.com/users/JINO-ROHIT/starred{/owner}{/repo}
https://api.github.com/users/JINO-ROHIT/subscriptions
User
https://api.github.com/users/JINO-ROHIT
public
COLLABORATOR
Yes, feel free to work on this :)
2024-04-03T18:06:33Z
https://github.com/huggingface/datasets/issues/6152#issuecomment-2035268700
https://api.github.com/repos/huggingface/datasets/issues/6152
IC_kwDODunzps55T7xc
https://api.github.com/repos/huggingface/datasets/issues/comments/2035268700/reactions
2024-04-03T18:06:33Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2035268700
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
CONTRIBUTOR
i think its working as expected . Heres the log i get for the same line -

![image](https://github.com/huggingface/datasets/assets/63234112/8a857ec5-8dd0-4b01-b3a7-7c93444b9558)

2024-04-04T11:45:28Z
https://github.com/huggingface/datasets/issues/6152#issuecomment-2036948225
https://api.github.com/repos/huggingface/datasets/issues/6152
IC_kwDODunzps55aV0B
https://api.github.com/repos/huggingface/datasets/issues/comments/2036948225/reactions
2024-04-04T11:45:28Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2036948225
https://avatars.githubusercontent.com/u/63234112?v=4
https://api.github.com/users/JINO-ROHIT/events{/privacy}
https://api.github.com/users/JINO-ROHIT/followers
https://api.github.com/users/JINO-ROHIT/following{/other_user}
https://api.github.com/users/JINO-ROHIT/gists{/gist_id}
https://github.com/JINO-ROHIT
JINO-ROHIT
MDQ6VXNlcjYzMjM0MTEy
https://api.github.com/users/JINO-ROHIT/orgs
https://api.github.com/users/JINO-ROHIT/received_events
https://api.github.com/users/JINO-ROHIT/repos
https://api.github.com/users/JINO-ROHIT/starred{/owner}{/repo}
https://api.github.com/users/JINO-ROHIT/subscriptions
User
https://api.github.com/users/JINO-ROHIT
public
NONE
#self-assign
2024-12-09T15:46:44Z
https://github.com/huggingface/datasets/issues/6152#issuecomment-2528456227
https://api.github.com/repos/huggingface/datasets/issues/6152
IC_kwDODunzps6WtS4j
https://api.github.com/repos/huggingface/datasets/issues/comments/2528456227/reactions
2024-12-09T15:46:44Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2528456227
https://avatars.githubusercontent.com/u/20582290?v=4
https://api.github.com/users/sahillihas/events{/privacy}
https://api.github.com/users/sahillihas/followers
https://api.github.com/users/sahillihas/following{/other_user}
https://api.github.com/users/sahillihas/gists{/gist_id}
https://github.com/sahillihas
sahillihas
MDQ6VXNlcjIwNTgyMjkw
https://api.github.com/users/sahillihas/orgs
https://api.github.com/users/sahillihas/received_events
https://api.github.com/users/sahillihas/repos
https://api.github.com/users/sahillihas/starred{/owner}{/repo}
https://api.github.com/users/sahillihas/subscriptions
User
https://api.github.com/users/sahillihas
public
NONE
Hey! I saw that PR #7314 fixed the issue when no matching data files are found, which is great. But I think the builder still defaults to the current directory when `data_dir` isn't specified, which could lead to unexpected behavior.

If no one’s actively working on that part, I’d be happy to pick it up and propose a fix. Let me know if that works!

2025-04-08T03:59:10Z
https://github.com/huggingface/datasets/issues/6152#issuecomment-2785160232
https://api.github.com/repos/huggingface/datasets/issues/6152
IC_kwDODunzps6mAiwo
https://api.github.com/repos/huggingface/datasets/issues/comments/2785160232/reactions
2025-04-08T03:59:10Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2785160232
https://avatars.githubusercontent.com/u/51700744?v=4
https://api.github.com/users/aniyagnik/events{/privacy}
https://api.github.com/users/aniyagnik/followers
https://api.github.com/users/aniyagnik/following{/other_user}
https://api.github.com/users/aniyagnik/gists{/gist_id}
https://github.com/aniyagnik
aniyagnik
MDQ6VXNlcjUxNzAwNzQ0
https://api.github.com/users/aniyagnik/orgs
https://api.github.com/users/aniyagnik/received_events
https://api.github.com/users/aniyagnik/repos
https://api.github.com/users/aniyagnik/starred{/owner}{/repo}
https://api.github.com/users/aniyagnik/subscriptions
User
https://api.github.com/users/aniyagnik
public
MEMBER
Cool, feel free to ping me if you open a PR @aniyagnik :)
2025-04-08T13:13:01Z
https://github.com/huggingface/datasets/issues/6152#issuecomment-2786398140
https://api.github.com/repos/huggingface/datasets/issues/6152
IC_kwDODunzps6mFQ-8
https://api.github.com/repos/huggingface/datasets/issues/comments/2786398140/reactions
2025-04-08T13:13:01Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2786398140
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Hey! I noticed this is still open and unresolved. I’d like to take this up and submit a PR to prevent resolving to the current directory when data_dir is not specified!!!
2025-06-16T07:34:04Z
https://github.com/huggingface/datasets/issues/6152#issuecomment-2975416212
https://api.github.com/repos/huggingface/datasets/issues/6152
IC_kwDODunzps6xWT-U
https://api.github.com/repos/huggingface/datasets/issues/comments/2975416212/reactions
2025-06-16T07:34:04Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2975416212
https://avatars.githubusercontent.com/u/142811259?v=4
https://api.github.com/users/ArjunJagdale/events{/privacy}
https://api.github.com/users/ArjunJagdale/followers
https://api.github.com/users/ArjunJagdale/following{/other_user}
https://api.github.com/users/ArjunJagdale/gists{/gist_id}
https://github.com/ArjunJagdale
ArjunJagdale
U_kgDOCIMgew
https://api.github.com/users/ArjunJagdale/orgs
https://api.github.com/users/ArjunJagdale/received_events
https://api.github.com/users/ArjunJagdale/repos
https://api.github.com/users/ArjunJagdale/starred{/owner}{/repo}
https://api.github.com/users/ArjunJagdale/subscriptions
User
https://api.github.com/users/ArjunJagdale
public
CONTRIBUTOR
Hi @aniyagnik @npuichigo @sahillihas @lhoestq @mariosasko,
I've opened a PR to address the issue discussed in #6152 by adding an early validation check for folder-based datasets. Would appreciate your review whenever convenient. Thanks!

https://github.com/huggingface/datasets/pull/7618
2025-06-16T07:54:09Z
https://github.com/huggingface/datasets/issues/6152#issuecomment-2975472110
https://api.github.com/repos/huggingface/datasets/issues/6152
IC_kwDODunzps6xWhnu
https://api.github.com/repos/huggingface/datasets/issues/comments/2975472110/reactions
2025-06-16T07:54:09Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2975472110
https://avatars.githubusercontent.com/u/142811259?v=4
https://api.github.com/users/ArjunJagdale/events{/privacy}
https://api.github.com/users/ArjunJagdale/followers
https://api.github.com/users/ArjunJagdale/following{/other_user}
https://api.github.com/users/ArjunJagdale/gists{/gist_id}
https://github.com/ArjunJagdale
ArjunJagdale
U_kgDOCIMgew
https://api.github.com/users/ArjunJagdale/orgs
https://api.github.com/users/ArjunJagdale/received_events
https://api.github.com/users/ArjunJagdale/repos
https://api.github.com/users/ArjunJagdale/starred{/owner}{/repo}
https://api.github.com/users/ArjunJagdale/subscriptions
User
https://api.github.com/users/ArjunJagdale
public
### Describe the bug

FolderBase Dataset automatically resolves under current directory when data_dir is not specified.

For example:
```
load_dataset("audiofolder")
```
takes long time to resolve and collect data_files from current directory. But I think it should reach out to this line for error handling https://github.com/huggingface/datasets/blob/cb8c5de5145c7e7eee65391cb7f4d92f0d565d62/src/datasets/packaged_modules/folder_based_builder/folder_based_builder.py#L58-L59

### Steps to reproduce the bug

```
load_dataset("audiofolder")
```

### Expected behavior

Error report

### Environment info

- `datasets` version: 2.14.4
- Platform: Linux-5.15.0-78-generic-x86_64-with-glibc2.17
- Python version: 3.8.15
- Huggingface_hub version: 0.16.4
- PyArrow version: 12.0.1
- Pandas version: 1.5.3