Trouble loading files defined in YAML explicitly
COLLABORATOR
There is a typo in one of the file names - `data/edf.csv` should be renamed to `data/def.csv` 🙂. 
2023-11-22T16:57:33Z
https://github.com/huggingface/datasets/issues/6443#issuecomment-1823147453
https://api.github.com/repos/huggingface/datasets/issues/6443
IC_kwDODunzps5sqwW9
https://api.github.com/repos/huggingface/datasets/issues/comments/1823147453/reactions
2023-11-22T16:57:33Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1823147453
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
COLLABORATOR
wow, I reviewed it twice to avoid being ashamed like that, but... I didn't notice the typo.

---

Besides this: do you think we would be able to improve the error message to make this clearer?
2023-11-23T08:44:07Z
https://github.com/huggingface/datasets/issues/6443#issuecomment-1823997584
https://api.github.com/repos/huggingface/datasets/issues/6443
IC_kwDODunzps5st_6Q
https://api.github.com/repos/huggingface/datasets/issues/comments/1823997584/reactions
2023-11-23T09:06:20Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1823997584
https://avatars.githubusercontent.com/u/1676121?v=4
https://api.github.com/users/severo/events{/privacy}
https://api.github.com/users/severo/followers
https://api.github.com/users/severo/following{/other_user}
https://api.github.com/users/severo/gists{/gist_id}
https://github.com/severo
severo
MDQ6VXNlcjE2NzYxMjE=
https://api.github.com/users/severo/orgs
https://api.github.com/users/severo/received_events
https://api.github.com/users/severo/repos
https://api.github.com/users/severo/starred{/owner}{/repo}
https://api.github.com/users/severo/subscriptions
User
https://api.github.com/users/severo
public
CONTRIBUTOR
Hi @severo 👋🏼

I'd like to work on improving the error message when files listed in `data_files` inside the YAML config are not found in the dataset repository.

My plan is to:
- Detect missing files explicitly by comparing listed paths in `data_files` with actual files in the repo
- Update the raised error to clearly mention which files are missing, e.g.:
  `"The following files were not found in the dataset repository: data/def.csv"`

Let me know if this direction makes sense — happy to start working on a PR!

2025-06-22T06:35:40Z
https://github.com/huggingface/datasets/issues/6443#issuecomment-2993972677
https://api.github.com/repos/huggingface/datasets/issues/6443
IC_kwDODunzps6ydGXF
https://api.github.com/repos/huggingface/datasets/issues/comments/2993972677/reactions
2025-06-22T06:35:40Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2993972677
https://avatars.githubusercontent.com/u/142811259?v=4
https://api.github.com/users/ArjunJagdale/events{/privacy}
https://api.github.com/users/ArjunJagdale/followers
https://api.github.com/users/ArjunJagdale/following{/other_user}
https://api.github.com/users/ArjunJagdale/gists{/gist_id}
https://github.com/ArjunJagdale
ArjunJagdale
U_kgDOCIMgew
https://api.github.com/users/ArjunJagdale/orgs
https://api.github.com/users/ArjunJagdale/received_events
https://api.github.com/users/ArjunJagdale/repos
https://api.github.com/users/ArjunJagdale/starred{/owner}{/repo}
https://api.github.com/users/ArjunJagdale/subscriptions
User
https://api.github.com/users/ArjunJagdale
public
COLLABORATOR
cc @lhoestq 
2025-06-23T08:33:38Z
https://github.com/huggingface/datasets/issues/6443#issuecomment-2995462771
https://api.github.com/repos/huggingface/datasets/issues/6443
IC_kwDODunzps6yiyJz
https://api.github.com/repos/huggingface/datasets/issues/comments/2995462771/reactions
2025-06-23T08:33:38Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2995462771
https://avatars.githubusercontent.com/u/1676121?v=4
https://api.github.com/users/severo/events{/privacy}
https://api.github.com/users/severo/followers
https://api.github.com/users/severo/following{/other_user}
https://api.github.com/users/severo/gists{/gist_id}
https://github.com/severo
severo
MDQ6VXNlcjE2NzYxMjE=
https://api.github.com/users/severo/orgs
https://api.github.com/users/severo/received_events
https://api.github.com/users/severo/repos
https://api.github.com/users/severo/starred{/owner}{/repo}
https://api.github.com/users/severo/subscriptions
User
https://api.github.com/users/severo
public
MEMBER
Hi ! Doesn't it raise a `FileNotFoundError` when it's the case ? see in the source code:

https://github.com/huggingface/datasets/blob/4a637bd963557bca42d3a2d8c54200fccb7ab91e/src/datasets/data_files.py#L379-L383
2025-06-23T12:46:28Z
https://github.com/huggingface/datasets/issues/6443#issuecomment-2996362852
https://api.github.com/repos/huggingface/datasets/issues/6443
IC_kwDODunzps6ymN5k
https://api.github.com/repos/huggingface/datasets/issues/comments/2996362852/reactions
2025-06-23T12:46:28Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2996362852
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Hi @lhoestq, thanks a lot for the pointer and for taking the time to review!

You're totally right — the FileNotFoundError does get raised from data_files.py when a file pattern isn't found.

What I was hoping to improve is the clarity of the error message, especially when users define multiple paths in data_files via a YAML config. For example, instead of the generic:

"Unable to find 'data/def.csv'"

it could say something like:

"The following files listed in the data_files YAML config were not found in the dataset repository: data/def.csv"

This could help users quickly catch typos or misnamed files in longer configs (like what happened in this issue 😅).

If that sounds like a helpful addition, I'd love to work on a small PR to enhance the error messaging.

Thanks again for all your help and feedback!
2025-06-23T13:46:46Z
https://github.com/huggingface/datasets/issues/6443#issuecomment-2996570664
https://api.github.com/repos/huggingface/datasets/issues/6443
IC_kwDODunzps6ynAoo
https://api.github.com/repos/huggingface/datasets/issues/comments/2996570664/reactions
2025-06-23T13:46:46Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2996570664
https://avatars.githubusercontent.com/u/142811259?v=4
https://api.github.com/users/ArjunJagdale/events{/privacy}
https://api.github.com/users/ArjunJagdale/followers
https://api.github.com/users/ArjunJagdale/following{/other_user}
https://api.github.com/users/ArjunJagdale/gists{/gist_id}
https://github.com/ArjunJagdale
ArjunJagdale
U_kgDOCIMgew
https://api.github.com/users/ArjunJagdale/orgs
https://api.github.com/users/ArjunJagdale/received_events
https://api.github.com/users/ArjunJagdale/repos
https://api.github.com/users/ArjunJagdale/starred{/owner}{/repo}
https://api.github.com/users/ArjunJagdale/subscriptions
User
https://api.github.com/users/ArjunJagdale
public
Look at https://huggingface.co/datasets/severo/doc-yaml-2

It's a reproduction of the example given in the docs at https://huggingface.co/docs/hub/datasets-manual-configuration

```
You can select multiple files per split using a list of paths:

my_dataset_repository/
├── README.md
├── data/
│   ├── abc.csv
│   └── def.csv
└── holdout/
    └── ghi.csv


---
configs:
- config_name: default
  data_files:
  - split: train
    path:
    - "data/abc.csv"
    - "data/def.csv"
  - split: test
    path: "holdout/ghi.csv"
---
```

It raises the following error:

```
Error code:   ConfigNamesError
Exception:    FileNotFoundError
Message:      Couldn't find a dataset script at /src/services/worker/severo/doc-yaml-2/doc-yaml-2.py or any data file in the same directory. Couldn't find 'severo/doc-yaml-2' on the Hugging Face Hub either: FileNotFoundError: Unable to find 'hf://datasets/severo/doc-yaml-2@938a0578fb4c6bc9da7d80b06a3ba39c2834b0c2/data/def.csv' with any supported extension ['.csv', '.tsv', '.json', '.jsonl', '.parquet', '.arrow', '.txt', '.blp', '.bmp', '.dib', '.bufr', '.cur', '.pcx', '.dcx', '.dds', '.ps', '.eps', '.fit', '.fits', '.fli', '.flc', '.ftc', '.ftu', '.gbr', '.gif', '.grib', '.h5', '.hdf', '.png', '.apng', '.jp2', '.j2k', '.jpc', '.jpf', '.jpx', '.j2c', '.icns', '.ico', '.im', '.iim', '.tif', '.tiff', '.jfif', '.jpe', '.jpg', '.jpeg', '.mpg', '.mpeg', '.msp', '.pcd', '.pxr', '.pbm', '.pgm', '.ppm', '.pnm', '.psd', '.bw', '.rgb', '.rgba', '.sgi', '.ras', '.tga', '.icb', '.vda', '.vst', '.webp', '.wmf', '.emf', '.xbm', '.xpm', '.BLP', '.BMP', '.DIB', '.BUFR', '.CUR', '.PCX', '.DCX', '.DDS', '.PS', '.EPS', '.FIT', '.FITS', '.FLI', '.FLC', '.FTC', '.FTU', '.GBR', '.GIF', '.GRIB', '.H5', '.HDF', '.PNG', '.APNG', '.JP2', '.J2K', '.JPC', '.JPF', '.JPX', '.J2C', '.ICNS', '.ICO', '.IM', '.IIM', '.TIF', '.TIFF', '.JFIF', '.JPE', '.JPG', '.JPEG', '.MPG', '.MPEG', '.MSP', '.PCD', '.PXR', '.PBM', '.PGM', '.PPM', '.PNM', '.PSD', '.BW', '.RGB', '.RGBA', '.SGI', '.RAS', '.TGA', '.ICB', '.VDA', '.VST', '.WEBP', '.WMF', '.EMF', '.XBM', '.XPM', '.aiff', '.au', '.avr', '.caf', '.flac', '.htk', '.svx', '.mat4', '.mat5', '.mpc2k', '.ogg', '.paf', '.pvf', '.raw', '.rf64', '.sd2', '.sds', '.ircam', '.voc', '.w64', '.wav', '.nist', '.wavex', '.wve', '.xi', '.mp3', '.opus', '.AIFF', '.AU', '.AVR', '.CAF', '.FLAC', '.HTK', '.SVX', '.MAT4', '.MAT5', '.MPC2K', '.OGG', '.PAF', '.PVF', '.RAW', '.RF64', '.SD2', '.SDS', '.IRCAM', '.VOC', '.W64', '.WAV', '.NIST', '.WAVEX', '.WVE', '.XI', '.MP3', '.OPUS', '.zip']
Traceback:    Traceback (most recent call last):
                File "/src/services/worker/src/worker/job_runners/dataset/config_names.py", line 65, in compute_config_names_response
                  for config in sorted(get_dataset_config_names(path=dataset, token=hf_token))
                File "/src/services/worker/.venv/lib/python3.9/site-packages/datasets/inspect.py", line 351, in get_dataset_config_names
                  dataset_module = dataset_module_factory(
                File "/src/services/worker/.venv/lib/python3.9/site-packages/datasets/load.py", line 1507, in dataset_module_factory
                  raise FileNotFoundError(
              FileNotFoundError: Couldn't find a dataset script at /src/services/worker/severo/doc-yaml-2/doc-yaml-2.py or any data file in the same directory. Couldn't find 'severo/doc-yaml-2' on the Hugging Face Hub either: FileNotFoundError: Unable to find 'hf://datasets/severo/doc-yaml-2@938a0578fb4c6bc9da7d80b06a3ba39c2834b0c2/data/def.csv' with any supported extension ['.csv', '.tsv', '.json', '.jsonl', '.parquet', '.arrow', '.txt', '.blp', '.bmp', '.dib', '.bufr', '.cur', '.pcx', '.dcx', '.dds', '.ps', '.eps', '.fit', '.fits', '.fli', '.flc', '.ftc', '.ftu', '.gbr', '.gif', '.grib', '.h5', '.hdf', '.png', '.apng', '.jp2', '.j2k', '.jpc', '.jpf', '.jpx', '.j2c', '.icns', '.ico', '.im', '.iim', '.tif', '.tiff', '.jfif', '.jpe', '.jpg', '.jpeg', '.mpg', '.mpeg', '.msp', '.pcd', '.pxr', '.pbm', '.pgm', '.ppm', '.pnm', '.psd', '.bw', '.rgb', '.rgba', '.sgi', '.ras', '.tga', '.icb', '.vda', '.vst', '.webp', '.wmf', '.emf', '.xbm', '.xpm', '.BLP', '.BMP', '.DIB', '.BUFR', '.CUR', '.PCX', '.DCX', '.DDS', '.PS', '.EPS', '.FIT', '.FITS', '.FLI', '.FLC', '.FTC', '.FTU', '.GBR', '.GIF', '.GRIB', '.H5', '.HDF', '.PNG', '.APNG', '.JP2', '.J2K', '.JPC', '.JPF', '.JPX', '.J2C', '.ICNS', '.ICO', '.IM', '.IIM', '.TIF', '.TIFF', '.JFIF', '.JPE', '.JPG', '.JPEG', '.MPG', '.MPEG', '.MSP', '.PCD', '.PXR', '.PBM', '.PGM', '.PPM', '.PNM', '.PSD', '.BW', '.RGB', '.RGBA', '.SGI', '.RAS', '.TGA', '.ICB', '.VDA', '.VST', '.WEBP', '.WMF', '.EMF', '.XBM', '.XPM', '.aiff', '.au', '.avr', '.caf', '.flac', '.htk', '.svx', '.mat4', '.mat5', '.mpc2k', '.ogg', '.paf', '.pvf', '.raw', '.rf64', '.sd2', '.sds', '.ircam', '.voc', '.w64', '.wav', '.nist', '.wavex', '.wve', '.xi', '.mp3', '.opus', '.AIFF', '.AU', '.AVR', '.CAF', '.FLAC', '.HTK', '.SVX', '.MAT4', '.MAT5', '.MPC2K', '.OGG', '.PAF', '.PVF', '.RAW', '.RF64', '.SD2', '.SDS', '.IRCAM', '.VOC', '.W64', '.WAV', '.NIST', '.WAVEX', '.WVE', '.XI', '.MP3', '.OPUS', '.zip']
```