[Multi Configs] Use directories to differentiate between subsets/configurations
NONE
I want to be able to create folders in a model.
2022-12-21T13:23:13Z
https://github.com/huggingface/datasets/issues/4578#issuecomment-1361308223
https://api.github.com/repos/huggingface/datasets/issues/4578
IC_kwDODunzps5RI-o_
https://api.github.com/repos/huggingface/datasets/issues/comments/1361308223/reactions
2022-12-21T13:23:13Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1361308223
https://avatars.githubusercontent.com/u/31076722?v=4
https://api.github.com/users/Jakeukalane/events{/privacy}
https://api.github.com/users/Jakeukalane/followers
https://api.github.com/users/Jakeukalane/following{/other_user}
https://api.github.com/users/Jakeukalane/gists{/gist_id}
https://github.com/Jakeukalane
Jakeukalane
MDQ6VXNlcjMxMDc2NzIy
https://api.github.com/users/Jakeukalane/orgs
https://api.github.com/users/Jakeukalane/received_events
https://api.github.com/users/Jakeukalane/repos
https://api.github.com/users/Jakeukalane/starred{/owner}{/repo}
https://api.github.com/users/Jakeukalane/subscriptions
User
https://api.github.com/users/Jakeukalane
public
NONE
How to set new split names, instead of train/test/validation? For example, I have a local dataset, consists of several subsets, named "A", "B", and "C". How can I create a huggingface dataset, with splits A/B/C ?

The document in https://huggingface.co/docs/datasets/dataset_script only tells me how to create datasets with subsets that is hosted on another server. How to do it if my datasets are local?
2023-01-03T04:01:32Z
https://github.com/huggingface/datasets/issues/4578#issuecomment-1369374270
https://api.github.com/repos/huggingface/datasets/issues/4578
IC_kwDODunzps5Rnv4-
https://api.github.com/repos/huggingface/datasets/issues/comments/1369374270/reactions
2023-01-03T04:01:32Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1369374270
https://avatars.githubusercontent.com/u/37113676?v=4
https://api.github.com/users/beyondguo/events{/privacy}
https://api.github.com/users/beyondguo/followers
https://api.github.com/users/beyondguo/following{/other_user}
https://api.github.com/users/beyondguo/gists{/gist_id}
https://github.com/beyondguo
beyondguo
MDQ6VXNlcjM3MTEzNjc2
https://api.github.com/users/beyondguo/orgs
https://api.github.com/users/beyondguo/received_events
https://api.github.com/users/beyondguo/repos
https://api.github.com/users/beyondguo/starred{/owner}{/repo}
https://api.github.com/users/beyondguo/subscriptions
User
https://api.github.com/users/beyondguo
public
MEMBER
> The document in https://huggingface.co/docs/datasets/dataset_script only tells me how to create datasets with subsets that is hosted on another server. How to do it if my datasets are local?

It works the same - you just need to use local paths instead of URLs
2023-01-03T11:18:41Z
https://github.com/huggingface/datasets/issues/4578#issuecomment-1369652776
https://api.github.com/repos/huggingface/datasets/issues/4578
IC_kwDODunzps5Roz4o
https://api.github.com/repos/huggingface/datasets/issues/comments/1369652776/reactions
2023-01-03T11:18:41Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1369652776
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
Currently to define several subsets/configurations of your dataset, you need to use a dataset script.

However it would be nice to have a no-code way to to this. 

For example we could specify different configurations of a dataset (for example, if a dataset contains different languages) with one directory per configuration.

These structures are not supported right now, but would be nice to have:


```
my_dataset_repository/
├── README.md
├── en/
│   ├── train.csv
│   └── test.csv
└── fr/
    ├── train.csv
    └── test.csv
```

Or with one directory per split:

```
my_dataset_repository/
├── README.md
├── en/
│   ├── train/
│   │   ├── shard_0.csv
│   │   └── shard_1.csv
│   └── test/
│       ├── shard_0.csv
│       └── shard_1.csv
└── fr/
    ├── train/
    │   ├── shard_0.csv
    │   └── shard_1.csv
    └── test/
        ├── shard_0.csv
        └── shard_1.csv
```

cc @stevhliu @albertvillanova 

This can be specified in the README as YAML with

```
configs:
- config_name: en
  data_dir: en
- config_name: fr
  data_dir: fr
```