Reduce friction in tabular dataset workflow by eliminating having splits when dataset is loaded
COLLABORATOR
I have to admit I'm not a fan of this idea, as this would result in a non-consistent behavior between tabular and non-tabular datasets, which is confusing if done without the context you provided. Instead, we could consider returning a `Dataset` object rather than `DatasetDict` if there is only one split in the generated dataset. But then again, I think this lib is a bit too old to make such changes. @lhoestq @albertvillanova WDYT?


2022-11-03T16:53:05Z
https://github.com/huggingface/datasets/issues/5189#issuecomment-1302397585
https://api.github.com/repos/huggingface/datasets/issues/5189
IC_kwDODunzps5NoQKR
https://api.github.com/repos/huggingface/datasets/issues/comments/1302397585/reactions
2022-11-03T16:53:05Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1302397585
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
MEMBER
We can brainstorm here to see how we could make it happen ? And then depending on the options we see if it's a change we can do.

I'm starting with a first reasoning

Currently not passing `split=` in `load_dataset` means "return a dict with each split".

Now what would happen if a dataset has no split ? Ideally it should return one Dataset. And passing `split=` would have no sense. So depending on the dataset content, not passing `split=` should return a dict or a Dataset. In particular, those two cases should work:
```python
# case 1: dataset without split
ds = load_dataset("dataset_without_split")
ds[0], ds["column_name"], list(ds) # we want this

# case 2: dataset with splits
ds = load_dataset("dataset_with_splits")
ds["train"] # this works and can't be changed
ds = load_dataset("dataset_with_splits", split="train")
ds[0], ds["column_name"], list(ds) # this works and can't be changed
```

I can see several ideas:
1. allowing `load_dataset` to return a different object based on the dataset content - either a Dataset or a DatasetDict
    - we can update `get_dataset_split_names` to return None or a list if users want to know in advance what object will be returned. They can also use `isinstance` _a posteriori_
    - but in this case we expect users to be careful when loading datasets and always to extra steps to check if they got a Dataset or DatasetDict
2. merge Dataset and DatasetDict objects
    - they already share many functions: map, filter, push_to_hub etc.
    - we can define `ds[0]` to be the first item of the first split, and consider that the uses accesses rows from the full table of all the splits concatenated
    - however there is a collision when doing `ds["column_name"]` or `ds["train"]` that we need to address: the first returns a list, while the other returns a Dataset.

What are your opinions on those two ideas ? Do you have other ideas in mind ?
2022-11-04T14:21:32Z
https://github.com/huggingface/datasets/issues/5189#issuecomment-1303628034
https://api.github.com/repos/huggingface/datasets/issues/5189
IC_kwDODunzps5Ns8kC
https://api.github.com/repos/huggingface/datasets/issues/comments/1303628034/reactions
2022-11-04T14:22:15Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1303628034
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
COLLABORATOR
I like the first idea more (concatenating splits doesn't seem useful, no?). This is a significant breaking change, so I think we should do a poll (or something similar) to gather more info on the actual "expected behavior" and wait for Datasets 3.0 if we decide to implement it.

PS: @thomwolf also suggested the same thing a while ago (https://github.com/huggingface/datasets/issues/743#issuecomment-746074641).
2022-11-04T14:40:39Z
https://github.com/huggingface/datasets/issues/5189#issuecomment-1303662698
https://api.github.com/repos/huggingface/datasets/issues/5189
IC_kwDODunzps5NtFBq
https://api.github.com/repos/huggingface/datasets/issues/comments/1303662698/reactions
2022-11-04T14:40:39Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1303662698
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
MEMBER
I think it's an interesting improvement to the user experience for a case that comes often (no split) so I would definitively support it.

I would be more in favor of option 2 rather than returning various types of objects from load_dataset and handling carefully the possible collisions indeed
2022-11-04T14:47:31Z
https://github.com/huggingface/datasets/issues/5189#issuecomment-1303673434
https://api.github.com/repos/huggingface/datasets/issues/5189
IC_kwDODunzps5NtHpa
https://api.github.com/repos/huggingface/datasets/issues/comments/1303673434/reactions
2022-11-04T14:47:31Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1303673434
https://avatars.githubusercontent.com/u/7353373?v=4
https://api.github.com/users/thomwolf/events{/privacy}
https://api.github.com/users/thomwolf/followers
https://api.github.com/users/thomwolf/following{/other_user}
https://api.github.com/users/thomwolf/gists{/gist_id}
https://github.com/thomwolf
thomwolf
MDQ6VXNlcjczNTMzNzM=
https://api.github.com/users/thomwolf/orgs
https://api.github.com/users/thomwolf/received_events
https://api.github.com/users/thomwolf/repos
https://api.github.com/users/thomwolf/starred{/owner}{/repo}
https://api.github.com/users/thomwolf/subscriptions
User
https://api.github.com/users/thomwolf
public
COLLABORATOR
Related: if a dataset only has one split, we don't show the splits select control in the dataset viewer on the Hub, eg. compare  https://huggingface.co/datasets/hf-internal-testing/fixtures_image_utils/viewer/image/test with https://huggingface.co/datasets/glue/viewer/mnli/test.

See https://github.com/huggingface/moon-landing/pull/3858 for more details (internal)
2022-11-04T15:51:54Z
https://github.com/huggingface/datasets/issues/5189#issuecomment-1303794112
https://api.github.com/repos/huggingface/datasets/issues/5189
IC_kwDODunzps5NtlHA
https://api.github.com/repos/huggingface/datasets/issues/comments/1303794112/reactions
2022-11-04T15:52:33Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1303794112
https://avatars.githubusercontent.com/u/1676121?v=4
https://api.github.com/users/severo/events{/privacy}
https://api.github.com/users/severo/followers
https://api.github.com/users/severo/following{/other_user}
https://api.github.com/users/severo/gists{/gist_id}
https://github.com/severo
severo
MDQ6VXNlcjE2NzYxMjE=
https://api.github.com/users/severo/orgs
https://api.github.com/users/severo/received_events
https://api.github.com/users/severo/repos
https://api.github.com/users/severo/starred{/owner}{/repo}
https://api.github.com/users/severo/subscriptions
User
https://api.github.com/users/severo
public
CONTRIBUTOR
I feel like the second idea is a bit more overkill. 
@severo I would say it's a bit irrelevant to the problem we have but is a separate problem @polinaeterna is solving at the moment. 😅 (also discussed on slack)
2022-11-07T10:36:32Z
https://github.com/huggingface/datasets/issues/5189#issuecomment-1305410096
https://api.github.com/repos/huggingface/datasets/issues/5189
IC_kwDODunzps5Nzvow
https://api.github.com/repos/huggingface/datasets/issues/comments/1305410096/reactions
2022-11-07T10:36:32Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1305410096
https://avatars.githubusercontent.com/u/53175384?v=4
https://api.github.com/users/merveenoyan/events{/privacy}
https://api.github.com/users/merveenoyan/followers
https://api.github.com/users/merveenoyan/following{/other_user}
https://api.github.com/users/merveenoyan/gists{/gist_id}
https://github.com/merveenoyan
merveenoyan
MDQ6VXNlcjUzMTc1Mzg0
https://api.github.com/users/merveenoyan/orgs
https://api.github.com/users/merveenoyan/received_events
https://api.github.com/users/merveenoyan/repos
https://api.github.com/users/merveenoyan/starred{/owner}{/repo}
https://api.github.com/users/merveenoyan/subscriptions
User
https://api.github.com/users/merveenoyan
public
COLLABORATOR
OK, sorry for polluting the thread. The relation I saw with the dataset viewer is that from a UX point of view, we hide the concepts of split and configuration whenever possible -> this issue feels like doing the same in the datasets library.
2022-11-07T10:40:40Z
https://github.com/huggingface/datasets/issues/5189#issuecomment-1305414607
https://api.github.com/repos/huggingface/datasets/issues/5189
IC_kwDODunzps5NzwvP
https://api.github.com/repos/huggingface/datasets/issues/comments/1305414607/reactions
2022-11-07T10:40:40Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1305414607
https://avatars.githubusercontent.com/u/1676121?v=4
https://api.github.com/users/severo/events{/privacy}
https://api.github.com/users/severo/followers
https://api.github.com/users/severo/following{/other_user}
https://api.github.com/users/severo/gists{/gist_id}
https://github.com/severo
severo
MDQ6VXNlcjE2NzYxMjE=
https://api.github.com/users/severo/orgs
https://api.github.com/users/severo/received_events
https://api.github.com/users/severo/repos
https://api.github.com/users/severo/starred{/owner}{/repo}
https://api.github.com/users/severo/subscriptions
User
https://api.github.com/users/severo
public
NONE
I would agree that returning different types based on the content of the dataset might be confusing.

We can do something similar to what `fetch_*` or `load_*` from `sklearn.datasets` do, which is to have an arg which changes the type of the returned type. For instance, `load_iris` would return a dict, but `load_iris(..., return_X_y=True)` would return a tuple.

Here we can have a similar arg such as `return_X` which would then only return a single `DataSet` or an array.
2022-11-07T11:07:22Z
https://github.com/huggingface/datasets/issues/5189#issuecomment-1305446265
https://api.github.com/repos/huggingface/datasets/issues/5189
IC_kwDODunzps5Nz4d5
https://api.github.com/repos/huggingface/datasets/issues/comments/1305446265/reactions
2022-11-07T11:07:22Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1305446265
https://avatars.githubusercontent.com/u/1663864?v=4
https://api.github.com/users/adrinjalali/events{/privacy}
https://api.github.com/users/adrinjalali/followers
https://api.github.com/users/adrinjalali/following{/other_user}
https://api.github.com/users/adrinjalali/gists{/gist_id}
https://github.com/adrinjalali
adrinjalali
MDQ6VXNlcjE2NjM4NjQ=
https://api.github.com/users/adrinjalali/orgs
https://api.github.com/users/adrinjalali/received_events
https://api.github.com/users/adrinjalali/repos
https://api.github.com/users/adrinjalali/starred{/owner}{/repo}
https://api.github.com/users/adrinjalali/subscriptions
User
https://api.github.com/users/adrinjalali
public
MEMBER
> I feel like the second idea is a bit more overkill.

Overkill in what sense ?

> Here we can have a similar arg such as return_X which would then only return a single DataSet or an array.

Right now one can already pass `split="all"` to get one `Dataset` object with all the data in it (unsplit). We could also have something like `return_all=True` so make the API clearer.

> I would be more in favor of option 2 rather than returning various types of objects from load_dataset and handling carefully the possible collisions indeed

I think it would be ok to handle the collision by allowing both `ds["train"]` and `ds["column_name"]` (and maybe adding something like `ds.splits` for those who want to iterate over the splits or add new ones)
2022-11-07T14:08:27Z
https://github.com/huggingface/datasets/issues/5189#issuecomment-1305670323
https://api.github.com/repos/huggingface/datasets/issues/5189
IC_kwDODunzps5N0vKz
https://api.github.com/repos/huggingface/datasets/issues/comments/1305670323/reactions
2022-11-07T14:08:27Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1305670323
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Would it make sense to remove the notion of "split" in `load_dataset`? I feel a lof of it comes from the want to have some sort of group of more or less similar dataset. "train"/"test"/"validation" are the traditional ones, but there are some datasets that have much more splits.

Would it make sense to force `load_dataset` to only load a single `Dataset` object, and fail if it doesn't point to one. And have another method that's like `load_dataset_group_info` that can return a very arbitrary info class (Dict, List whatever), but you need to pass individual infos to `load_dataset` to run anything? Typically I don't think `DatasetDict.map` is really that helpful, but that's my personal opinion. This would help make things more readable (typically knowing if an object is a `Dataset` or a `DatasetDict`)
2022-11-08T12:43:41Z
https://github.com/huggingface/datasets/issues/5189#issuecomment-1307158069
https://api.github.com/repos/huggingface/datasets/issues/5189
IC_kwDODunzps5N6aY1
https://api.github.com/repos/huggingface/datasets/issues/comments/1307158069/reactions
2022-11-08T12:43:41Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1307158069
https://avatars.githubusercontent.com/u/24695242?v=4
https://api.github.com/users/thomasw21/events{/privacy}
https://api.github.com/users/thomasw21/followers
https://api.github.com/users/thomasw21/following{/other_user}
https://api.github.com/users/thomasw21/gists{/gist_id}
https://github.com/thomasw21
thomasw21
MDQ6VXNlcjI0Njk1MjQy
https://api.github.com/users/thomasw21/orgs
https://api.github.com/users/thomasw21/received_events
https://api.github.com/users/thomasw21/repos
https://api.github.com/users/thomasw21/starred{/owner}{/repo}
https://api.github.com/users/thomasw21/subscriptions
User
https://api.github.com/users/thomasw21
public
MEMBER
> Would it make sense to remove the notion of "split" in load_dataset?

I think we need to keep it - though in practice people can name the splits whatever they want anyway.

> Would it make sense to force load_dataset to only load a single Dataset object, and fail if it doesn't point to one.

We need to keep backward compatibility ideally - in particular the load_dataset + ds["train"] one
2022-11-08T13:44:07Z
https://github.com/huggingface/datasets/issues/5189#issuecomment-1307244195
https://api.github.com/repos/huggingface/datasets/issues/5189
IC_kwDODunzps5N6vaj
https://api.github.com/repos/huggingface/datasets/issues/comments/1307244195/reactions
2022-11-08T13:44:07Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1307244195
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
> I think we need to keep it - though in practice people can name the splits whatever they want anyway.

It was my understanding that the whole issue was that `load_dataset` returned multiple types of objects.

> We need to keep backward compatibility ideally - in particular the load_dataset + ds["train"] one

Yeah sorry I meant ideally. One can always start developing `load_dataset_v2` can deprecate the first one and remove it in the longer term.
2022-11-08T13:54:03Z
https://github.com/huggingface/datasets/issues/5189#issuecomment-1307256673
https://api.github.com/repos/huggingface/datasets/issues/5189
IC_kwDODunzps5N6ydh
https://api.github.com/repos/huggingface/datasets/issues/comments/1307256673/reactions
2022-11-08T13:54:03Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1307256673
https://avatars.githubusercontent.com/u/24695242?v=4
https://api.github.com/users/thomasw21/events{/privacy}
https://api.github.com/users/thomasw21/followers
https://api.github.com/users/thomasw21/following{/other_user}
https://api.github.com/users/thomasw21/gists{/gist_id}
https://github.com/thomasw21
thomasw21
MDQ6VXNlcjI0Njk1MjQy
https://api.github.com/users/thomasw21/orgs
https://api.github.com/users/thomasw21/received_events
https://api.github.com/users/thomasw21/repos
https://api.github.com/users/thomasw21/starred{/owner}{/repo}
https://api.github.com/users/thomasw21/subscriptions
User
https://api.github.com/users/thomasw21
public
MEMBER
> It was my understanding that the whole issue was that load_dataset returned multiple types of objects.

Yes indeed, but we still want to keep a way to load the train/val/test/whatever splits alone ;)
2022-11-08T13:55:52Z
https://github.com/huggingface/datasets/issues/5189#issuecomment-1307258968
https://api.github.com/repos/huggingface/datasets/issues/5189
IC_kwDODunzps5N6zBY
https://api.github.com/repos/huggingface/datasets/issues/comments/1307258968/reactions
2022-11-08T13:55:52Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1307258968
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
@thomasw21's solution is good but it will break backwards compatibility. 😅
2022-11-08T18:52:29Z
https://github.com/huggingface/datasets/issues/5189#issuecomment-1307681317
https://api.github.com/repos/huggingface/datasets/issues/5189
IC_kwDODunzps5N8aIl
https://api.github.com/repos/huggingface/datasets/issues/comments/1307681317/reactions
2022-11-08T18:52:29Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1307681317
https://avatars.githubusercontent.com/u/53175384?v=4
https://api.github.com/users/merveenoyan/events{/privacy}
https://api.github.com/users/merveenoyan/followers
https://api.github.com/users/merveenoyan/following{/other_user}
https://api.github.com/users/merveenoyan/gists{/gist_id}
https://github.com/merveenoyan
merveenoyan
MDQ6VXNlcjUzMTc1Mzg0
https://api.github.com/users/merveenoyan/orgs
https://api.github.com/users/merveenoyan/received_events
https://api.github.com/users/merveenoyan/repos
https://api.github.com/users/merveenoyan/starred{/owner}{/repo}
https://api.github.com/users/merveenoyan/subscriptions
User
https://api.github.com/users/merveenoyan
public
MEMBER
Started to experiment with merging Dataset and DatasetDict. My plan is to define the splits of a Dataset in Dataset.info.splits (already exists, but never used). A Dataset would then be the concatenation of its splits if they exist.

Not sure yet this is the way to go. My plan is to play with it and see and share it with you, so we can see if it makes sense from a UX point of view.
2022-11-23T16:16:13Z
https://github.com/huggingface/datasets/issues/5189#issuecomment-1325327106
https://api.github.com/repos/huggingface/datasets/issues/5189
IC_kwDODunzps5O_uMC
https://api.github.com/repos/huggingface/datasets/issues/comments/1325327106/reactions
2022-11-23T16:16:13Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1325327106
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
So just to make sure that I understand the current direction, people will have to be extra careful when handling splits right?
Imagine "potato" a dataset containing train/validation split:
```
load_dataset("potato") # returns the concatenation of all the splits
```
Previously the design would force you to choose a split (it would raise otherwise), or manually concat them if you really wanted to play with concatenated splits. Now it would potentially run without raising for a bit of time until you figure out that you've been training on both train and validation split.

Would it make sense to use a dataset specific default instead of using the concatenation, typically "potato" dataset's default would be train?
```
load_dataset("potato") # returns "train" split
load_dataset("potato", split="train") # returns "train" split
load_dataset("potato", split="validation") # returns "validation" split
concatenate_datasets([load_dataset("potato", split="train"), load_dataset("potato", split="validation")]) # returns concatenation
```
2022-11-25T11:19:34Z
https://github.com/huggingface/datasets/issues/5189#issuecomment-1327353545
https://api.github.com/repos/huggingface/datasets/issues/5189
IC_kwDODunzps5PHc7J
https://api.github.com/repos/huggingface/datasets/issues/comments/1327353545/reactions
2022-11-25T11:22:40Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1327353545
https://avatars.githubusercontent.com/u/24695242?v=4
https://api.github.com/users/thomasw21/events{/privacy}
https://api.github.com/users/thomasw21/followers
https://api.github.com/users/thomasw21/following{/other_user}
https://api.github.com/users/thomasw21/gists{/gist_id}
https://github.com/thomasw21
thomasw21
MDQ6VXNlcjI0Njk1MjQy
https://api.github.com/users/thomasw21/orgs
https://api.github.com/users/thomasw21/received_events
https://api.github.com/users/thomasw21/repos
https://api.github.com/users/thomasw21/starred{/owner}{/repo}
https://api.github.com/users/thomasw21/subscriptions
User
https://api.github.com/users/thomasw21
public
MEMBER
> load_dataset("potato") # returns "train" split

To avoid a breaking change we need to be able to do `load_dataset("potato")["validation"]` as well.

In that case I'd wonder where the validation split comes from, since the rows of the dataset wouldn't contain the validation split according to your example. That's why I'm more in favor of concatenating.

A dataset is one table, that optionally has some split info about subsets (e.g. for training an evaluation)

This also allows anyone to re-split the dataset the way they want if they're not happy with the default:

```python
ds = load_dataset("potato").train_test_split(test_size=0.2)
train_ds = ds["train"]
test_ds = ds["test"]
```
2022-11-25T13:55:08Z
https://github.com/huggingface/datasets/issues/5189#issuecomment-1327504311
https://api.github.com/repos/huggingface/datasets/issues/5189
IC_kwDODunzps5PIBu3
https://api.github.com/repos/huggingface/datasets/issues/comments/1327504311/reactions
2022-11-25T13:55:08Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1327504311
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
Just thinking about this, we could just have `to_dataframe()` as `load_dataset("blah").to_dataframe()` to get the whole dataset, and not change anything else.
2022-11-25T17:17:39Z
https://github.com/huggingface/datasets/issues/5189#issuecomment-1327729362
https://api.github.com/repos/huggingface/datasets/issues/5189
IC_kwDODunzps5PI4rS
https://api.github.com/repos/huggingface/datasets/issues/comments/1327729362/reactions
2022-11-25T17:17:39Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1327729362
https://avatars.githubusercontent.com/u/1663864?v=4
https://api.github.com/users/adrinjalali/events{/privacy}
https://api.github.com/users/adrinjalali/followers
https://api.github.com/users/adrinjalali/following{/other_user}
https://api.github.com/users/adrinjalali/gists{/gist_id}
https://github.com/adrinjalali
adrinjalali
MDQ6VXNlcjE2NjM4NjQ=
https://api.github.com/users/adrinjalali/orgs
https://api.github.com/users/adrinjalali/received_events
https://api.github.com/users/adrinjalali/repos
https://api.github.com/users/adrinjalali/starred{/owner}{/repo}
https://api.github.com/users/adrinjalali/subscriptions
User
https://api.github.com/users/adrinjalali
public
MEMBER
I have a first implementation of option 2 (merging Dataset and DatasetDict) in this PR: https://github.com/huggingface/datasets/pull/5301/

Feel free to play with it if you're interested, and let me know what you think. In this PR, a dataset is one table that optionally has some split info about subsets.
2022-11-28T19:17:18Z
https://github.com/huggingface/datasets/issues/5189#issuecomment-1329621941
https://api.github.com/repos/huggingface/datasets/issues/5189
IC_kwDODunzps5PQGu1
https://api.github.com/repos/huggingface/datasets/issues/comments/1329621941/reactions
2022-11-28T19:17:18Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1329621941
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
@adrinjalali we already have [to_pandas](https://huggingface.co/docs/datasets/package_reference/main_classes#datasets.Dataset.to_pandas) AFAIK that essentially does the same thing (for a dataset, not for a dataset dict), I was wondering if it makes sense to have this as I don't know portion of people who load non-tabular datasets into dataframes. @lhoestq I saw your PR and it will break a lot of things imo, WDYT of this option? 
2022-11-29T11:02:16Z
https://github.com/huggingface/datasets/issues/5189#issuecomment-1330452727
https://api.github.com/repos/huggingface/datasets/issues/5189
IC_kwDODunzps5PTRj3
https://api.github.com/repos/huggingface/datasets/issues/comments/1330452727/reactions
2022-11-29T11:02:47Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1330452727
https://avatars.githubusercontent.com/u/53175384?v=4
https://api.github.com/users/merveenoyan/events{/privacy}
https://api.github.com/users/merveenoyan/followers
https://api.github.com/users/merveenoyan/following{/other_user}
https://api.github.com/users/merveenoyan/gists{/gist_id}
https://github.com/merveenoyan
merveenoyan
MDQ6VXNlcjUzMTc1Mzg0
https://api.github.com/users/merveenoyan/orgs
https://api.github.com/users/merveenoyan/received_events
https://api.github.com/users/merveenoyan/repos
https://api.github.com/users/merveenoyan/starred{/owner}{/repo}
https://api.github.com/users/merveenoyan/subscriptions
User
https://api.github.com/users/merveenoyan
public
MEMBER
> we already have [to_pandas](https://huggingface.co/docs/datasets/package_reference/main_classes#datasets.Dataset.to_pandas) AFAIK that essentially does the same thing (for a dataset, not for a dataset dict)

yes correct :)

> I saw your PR and it will break a lot of things imo

Do you have concrete examples you can share ?

> WDYT of this option?

The to_dataframe option ? I think it not enough, since you'd still get a `DatasetDict({"train": Dataset()})` if you load a dataset with no splits (e.g. one CSV), and this doesn't really make sense.

Note that in the PR I opened you can do
```python
ds = load_dataset("dataset_with_just_one_csv")  # Dataset type
df = load_dataset("dataset_with_just_one_csv").to_pandas()  # DataFrame type
```
2022-11-29T11:25:26Z
https://github.com/huggingface/datasets/issues/5189#issuecomment-1330479739
https://api.github.com/repos/huggingface/datasets/issues/5189
IC_kwDODunzps5PTYJ7
https://api.github.com/repos/huggingface/datasets/issues/comments/1330479739/reactions
2022-11-29T11:25:26Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1330479739
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
@lhoestq no I think @adrinjalali and I meant when user calls `to_dataframe` if there's only train split in `DatasetDict` we could directly load that into dataframe. This might cause a confusion given there's to_pandas but I think it's more intuitive and least breaking change. (given people -who use `datasets` for tabular workflows- will eventually call `to_pandas` anyway) 
2022-11-29T12:04:19Z
https://github.com/huggingface/datasets/issues/5189#issuecomment-1330523620
https://api.github.com/repos/huggingface/datasets/issues/5189
IC_kwDODunzps5PTi3k
https://api.github.com/repos/huggingface/datasets/issues/comments/1330523620/reactions
2022-11-29T12:04:28Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1330523620
https://avatars.githubusercontent.com/u/53175384?v=4
https://api.github.com/users/merveenoyan/events{/privacy}
https://api.github.com/users/merveenoyan/followers
https://api.github.com/users/merveenoyan/following{/other_user}
https://api.github.com/users/merveenoyan/gists{/gist_id}
https://github.com/merveenoyan
merveenoyan
MDQ6VXNlcjUzMTc1Mzg0
https://api.github.com/users/merveenoyan/orgs
https://api.github.com/users/merveenoyan/received_events
https://api.github.com/users/merveenoyan/repos
https://api.github.com/users/merveenoyan/starred{/owner}{/repo}
https://api.github.com/users/merveenoyan/subscriptions
User
https://api.github.com/users/merveenoyan
public
MEMBER
So in that case it would be fine to still end up with a dataset dict with a "train" split ?
2022-11-29T13:57:36Z
https://github.com/huggingface/datasets/issues/5189#issuecomment-1330690284
https://api.github.com/repos/huggingface/datasets/issues/5189
IC_kwDODunzps5PULjs
https://api.github.com/repos/huggingface/datasets/issues/comments/1330690284/reactions
2022-11-29T13:57:36Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1330690284
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
yeah what I mean is this:

```py
dataset = load_dataset("blah")

# deal with a split of the dataset
train = dataset["train"]
train_df = dataset["train"].to_dataframe()

# deal with the whole dataset
dataset_df = dataset.to_dataframe()
```

So we do two things to improve tabular experience:
- allow datasets to have a single split
- add `to_dataframe` to the root dict level so that users can simply call `df = load_dataset("blah").to_dataframe()` and have it in their `pandas.DataFrame` object.
2022-11-29T14:32:15Z
https://github.com/huggingface/datasets/issues/5189#issuecomment-1330739272
https://api.github.com/repos/huggingface/datasets/issues/5189
IC_kwDODunzps5PUXhI
https://api.github.com/repos/huggingface/datasets/issues/comments/1330739272/reactions
2022-11-29T14:32:15Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1330739272
https://avatars.githubusercontent.com/u/1663864?v=4
https://api.github.com/users/adrinjalali/events{/privacy}
https://api.github.com/users/adrinjalali/followers
https://api.github.com/users/adrinjalali/following{/other_user}
https://api.github.com/users/adrinjalali/gists{/gist_id}
https://github.com/adrinjalali
adrinjalali
MDQ6VXNlcjE2NjM4NjQ=
https://api.github.com/users/adrinjalali/orgs
https://api.github.com/users/adrinjalali/received_events
https://api.github.com/users/adrinjalali/repos
https://api.github.com/users/adrinjalali/starred{/owner}{/repo}
https://api.github.com/users/adrinjalali/subscriptions
User
https://api.github.com/users/adrinjalali
public
MEMBER
Ok ! Note that we already have `Dataset.to_pandas()` so for consistency I'd call it `DatasetDict.to_pandas()` as well, does it sound good to you ? This is something we can add pretty easily
2022-11-29T14:46:11Z
https://github.com/huggingface/datasets/issues/5189#issuecomment-1330760115
https://api.github.com/repos/huggingface/datasets/issues/5189
IC_kwDODunzps5PUcmz
https://api.github.com/repos/huggingface/datasets/issues/comments/1330760115/reactions
2022-11-29T14:46:32Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1330760115
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
yeah that sounds perfect @lhoestq !
2022-11-29T14:48:30Z
https://github.com/huggingface/datasets/issues/5189#issuecomment-1330764701
https://api.github.com/repos/huggingface/datasets/issues/5189
IC_kwDODunzps5PUdud
https://api.github.com/repos/huggingface/datasets/issues/comments/1330764701/reactions
2022-11-29T14:48:30Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1330764701
https://avatars.githubusercontent.com/u/1663864?v=4
https://api.github.com/users/adrinjalali/events{/privacy}
https://api.github.com/users/adrinjalali/followers
https://api.github.com/users/adrinjalali/following{/other_user}
https://api.github.com/users/adrinjalali/gists{/gist_id}
https://github.com/adrinjalali
adrinjalali
MDQ6VXNlcjE2NjM4NjQ=
https://api.github.com/users/adrinjalali/orgs
https://api.github.com/users/adrinjalali/received_events
https://api.github.com/users/adrinjalali/repos
https://api.github.com/users/adrinjalali/starred{/owner}{/repo}
https://api.github.com/users/adrinjalali/subscriptions
User
https://api.github.com/users/adrinjalali
public
MEMBER
> So just to make sure that I understand the current direction, people will have to be extra careful when handling splits right?

We can raise an error if someone does `load_dataset(...)[0]` if the dataset is made of several splits, and return the first example if there's one or zero splits (i.e. when it's not ambiguous). Had this idea from the dicussions in #5312 WDYT @thomasw21 ?
2022-12-01T16:18:49Z
https://github.com/huggingface/datasets/issues/5189#issuecomment-1334018198
https://api.github.com/repos/huggingface/datasets/issues/5189
IC_kwDODunzps5Pg4CW
https://api.github.com/repos/huggingface/datasets/issues/comments/1334018198/reactions
2022-12-01T16:18:55Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1334018198
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
> We can raise an error if someone does load_dataset(...)[0] if the dataset is made of several splits,

But then how is that different to have the distinction between DatasetDict and Dataset then? Is it just that "default behaviour when there are no splits or single split, it returns directly the split when there's no ambiguity".

Also I was wondering how the concatenation could have heavy impacts when running mapping functions/filtering in batch? Typically can batch be somehow mixed?
2022-12-02T13:50:12Z
https://github.com/huggingface/datasets/issues/5189#issuecomment-1335257983
https://api.github.com/repos/huggingface/datasets/issues/5189
IC_kwDODunzps5Plmt_
https://api.github.com/repos/huggingface/datasets/issues/comments/1335257983/reactions
2022-12-02T14:04:28Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1335257983
https://avatars.githubusercontent.com/u/24695242?v=4
https://api.github.com/users/thomasw21/events{/privacy}
https://api.github.com/users/thomasw21/followers
https://api.github.com/users/thomasw21/following{/other_user}
https://api.github.com/users/thomasw21/gists{/gist_id}
https://github.com/thomasw21
thomasw21
MDQ6VXNlcjI0Njk1MjQy
https://api.github.com/users/thomasw21/orgs
https://api.github.com/users/thomasw21/received_events
https://api.github.com/users/thomasw21/repos
https://api.github.com/users/thomasw21/starred{/owner}{/repo}
https://api.github.com/users/thomasw21/subscriptions
User
https://api.github.com/users/thomasw21
public
MEMBER
> But then how is that different to have the distinction between DatasetDict and Dataset then?

Because it doesn't make sense to be able to do `example = ds[0]` or `examples = list(ds)` on a class named `DatasetDict` of type `Dict[str, Dataset]`.

> Also I was wondering how the concatenation could have heavy impacts when running mapping functions/filtering in batch? Typically can batch be somehow mixed?

No, we run each function on each split separated
2022-12-02T14:05:41Z
https://github.com/huggingface/datasets/issues/5189#issuecomment-1335277359
https://api.github.com/repos/huggingface/datasets/issues/5189
IC_kwDODunzps5Plrcv
https://api.github.com/repos/huggingface/datasets/issues/comments/1335277359/reactions
2022-12-02T14:05:41Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1335277359
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
> Because it doesn't make sense to be able to do example = ds[0] or examples = list(ds) on a class named DatasetDict of type Dict[str, Dataset].

Hum but you're still going to raise an exception in both those cases with your current change no? (actually list(ds) would return the name of the splits no?)

> No, we run each function on each split separated

Nice!
2022-12-02T14:09:22Z
https://github.com/huggingface/datasets/issues/5189#issuecomment-1335282129
https://api.github.com/repos/huggingface/datasets/issues/5189
IC_kwDODunzps5PlsnR
https://api.github.com/repos/huggingface/datasets/issues/comments/1335282129/reactions
2022-12-02T14:09:31Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1335282129
https://avatars.githubusercontent.com/u/24695242?v=4
https://api.github.com/users/thomasw21/events{/privacy}
https://api.github.com/users/thomasw21/followers
https://api.github.com/users/thomasw21/following{/other_user}
https://api.github.com/users/thomasw21/gists{/gist_id}
https://github.com/thomasw21
thomasw21
MDQ6VXNlcjI0Njk1MjQy
https://api.github.com/users/thomasw21/orgs
https://api.github.com/users/thomasw21/received_events
https://api.github.com/users/thomasw21/repos
https://api.github.com/users/thomasw21/starred{/owner}{/repo}
https://api.github.com/users/thomasw21/subscriptions
User
https://api.github.com/users/thomasw21
public
### Feature request

Sorry for cryptic name but I'd like to explain using code itself. When I want to load a specific dataset from a repository (for instance, this: https://huggingface.co/datasets/inria-soda/tabular-benchmark)

```python
from datasets import load_dataset
dataset = load_dataset("inria-soda/tabular-benchmark", data_files=["reg_cat/house_sales.csv"], streaming=True)
print(next(iter(dataset["train"])))
```

`datasets` library is essentially designed for people who'd like to use benchmark datasets on various modalities to fine-tune their models, and these benchmark datasets usually have pre-defined train and test splits. However, for tabular workflows, having train and test splits usually ends up model overfitting to validation split so usually the users would like to do validation techniques like `StratifiedKFoldCrossValidation` or when they tune for hyperparameters they do `GridSearchCrossValidation` so often the behavior is to create their own splits. Even [in this paper](https://hal.archives-ouvertes.fr/hal-03723551) a benchmark is introduced but the split is done by authors.
It's a bit confusing for average tabular user to try and load a dataset and see `"train"` so it would be nice if we would not load dataset into a split called `train `by default.

```diff
from datasets import load_dataset
dataset = load_dataset("inria-soda/tabular-benchmark", data_files=["reg_cat/house_sales.csv"], streaming=True)
-print(next(iter(dataset["train"])))
+print(next(iter(dataset)))
``` 

### Motivation

I explained it above 😅

### Your contribution

I think this is quite a big change that seems small (e.g. how to determine datasets that will not be load to train split?), it's best if we discuss first!