Webdataset data format problem
NONE
I was able to work around it
2025-03-21T19:19:57Z
https://github.com/huggingface/datasets/issues/7473#issuecomment-2744255429
https://api.github.com/repos/huggingface/datasets/issues/7473
IC_kwDODunzps6jkgPF
https://api.github.com/repos/huggingface/datasets/issues/comments/2744255429/reactions
2025-03-21T19:19:57Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2744255429
https://avatars.githubusercontent.com/u/1017189?v=4
https://api.github.com/users/edmcman/events{/privacy}
https://api.github.com/users/edmcman/followers
https://api.github.com/users/edmcman/following{/other_user}
https://api.github.com/users/edmcman/gists{/gist_id}
https://github.com/edmcman
edmcman
MDQ6VXNlcjEwMTcxODk=
https://api.github.com/users/edmcman/orgs
https://api.github.com/users/edmcman/received_events
https://api.github.com/users/edmcman/repos
https://api.github.com/users/edmcman/starred{/owner}{/repo}
https://api.github.com/users/edmcman/subscriptions
User
https://api.github.com/users/edmcman
public
### Describe the bug

Please see https://huggingface.co/datasets/ejschwartz/idioms/discussions/1

Error code: FileFormatMismatchBetweenSplitsError

All three splits, train, test, and validation, use webdataset. But only the train split has more than one file. How can I force the other two splits to also be interpreted as being the webdataset format?  (I don't think there is currently a way, but happy to be told that I am wrong.)

### Steps to reproduce the bug

```
import datasets
datasets.load_dataset("ejschwartz/idioms")

### Expected behavior

The dataset loads.  Alternatively, there is a YAML syntax for manually specifying the format.

### Environment info

- `datasets` version: 3.2.0
- Platform: Linux-6.8.0-52-generic-x86_64-with-glibc2.35
- Python version: 3.10.12
- `huggingface_hub` version: 0.28.1
- PyArrow version: 19.0.0
- Pandas version: 2.2.3
- `fsspec` version: 2024.9.0