load_from_disk and save_to_disk are not compatible with each other
COLLABORATOR
Hi,

`load_dataset` returns an instance of `DatasetDict` if `split` is not specified, so instead of `Dataset.load_from_disk`, use `DatasetDict.load_from_disk` to load the dataset from disk.
2021-05-29T00:37:27Z
https://github.com/huggingface/datasets/issues/2424#issuecomment-850736820
https://api.github.com/repos/huggingface/datasets/issues/2424
MDEyOklzc3VlQ29tbWVudDg1MDczNjgyMA==
https://api.github.com/repos/huggingface/datasets/issues/comments/850736820/reactions
2021-05-29T00:37:27Z
https://api.github.com/repos/huggingface/datasets/issues/comments/850736820
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
NONE
Thanks it worked!
2021-05-31T17:12:15Z
https://github.com/huggingface/datasets/issues/2424#issuecomment-851603290
https://api.github.com/repos/huggingface/datasets/issues/2424
MDEyOklzc3VlQ29tbWVudDg1MTYwMzI5MA==
https://api.github.com/repos/huggingface/datasets/issues/comments/851603290/reactions
2021-05-31T17:12:15Z
https://api.github.com/repos/huggingface/datasets/issues/comments/851603290
https://avatars.githubusercontent.com/u/7584674?v=4
https://api.github.com/users/roholazandie/events{/privacy}
https://api.github.com/users/roholazandie/followers
https://api.github.com/users/roholazandie/following{/other_user}
https://api.github.com/users/roholazandie/gists{/gist_id}
https://github.com/roholazandie
roholazandie
MDQ6VXNlcjc1ODQ2NzQ=
https://api.github.com/users/roholazandie/orgs
https://api.github.com/users/roholazandie/received_events
https://api.github.com/users/roholazandie/repos
https://api.github.com/users/roholazandie/starred{/owner}{/repo}
https://api.github.com/users/roholazandie/subscriptions
User
https://api.github.com/users/roholazandie
public
MEMBER
Though I see a stream of issues open by people lost between datasets and datasets dicts so maybe there is here something that could be better in terms of UX. Could be better error handling or something else smarter to even avoid said errors but maybe we should think about this. Reopening to use this issue as a discussion place but feel free to open a new open if you prefer @lhoestq @albertvillanova 
2021-05-31T19:07:23Z
https://github.com/huggingface/datasets/issues/2424#issuecomment-851640509
https://api.github.com/repos/huggingface/datasets/issues/2424
MDEyOklzc3VlQ29tbWVudDg1MTY0MDUwOQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/851640509/reactions
2021-05-31T19:07:23Z
https://api.github.com/repos/huggingface/datasets/issues/comments/851640509
https://avatars.githubusercontent.com/u/7353373?v=4
https://api.github.com/users/thomwolf/events{/privacy}
https://api.github.com/users/thomwolf/followers
https://api.github.com/users/thomwolf/following{/other_user}
https://api.github.com/users/thomwolf/gists{/gist_id}
https://github.com/thomwolf
thomwolf
MDQ6VXNlcjczNTMzNzM=
https://api.github.com/users/thomwolf/orgs
https://api.github.com/users/thomwolf/received_events
https://api.github.com/users/thomwolf/repos
https://api.github.com/users/thomwolf/starred{/owner}{/repo}
https://api.github.com/users/thomwolf/subscriptions
User
https://api.github.com/users/thomwolf
public
MEMBER
We should probably improve the error message indeed.

Also note that there exists a function `load_from_disk` that can load a Dataset or a DatasetDict. Under the hood it calls either `Dataset.load_from_disk` or `DatasetDict.load_from_disk`:


```python
from datasets import load_from_disk

dataset_dict = load_from_disk("path/to/dataset/dict")
single_dataset = load_from_disk("path/to/single/dataset")
```
2021-06-01T08:58:10Z
https://github.com/huggingface/datasets/issues/2424#issuecomment-851956649
https://api.github.com/repos/huggingface/datasets/issues/2424
MDEyOklzc3VlQ29tbWVudDg1MTk1NjY0OQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/851956649/reactions
2021-06-01T08:58:10Z
https://api.github.com/repos/huggingface/datasets/issues/comments/851956649
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
I just opened #2437 to improve the error message
2021-06-01T09:44:54Z
https://github.com/huggingface/datasets/issues/2424#issuecomment-851987803
https://api.github.com/repos/huggingface/datasets/issues/2424
MDEyOklzc3VlQ29tbWVudDg1MTk4NzgwMw==
https://api.github.com/repos/huggingface/datasets/issues/comments/851987803/reactions
2021-06-01T09:44:54Z
https://api.github.com/repos/huggingface/datasets/issues/comments/851987803
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
Superseded by #2462 
2021-06-08T19:22:32Z
https://github.com/huggingface/datasets/issues/2424#issuecomment-857038837
https://api.github.com/repos/huggingface/datasets/issues/2424
MDEyOklzc3VlQ29tbWVudDg1NzAzODgzNw==
https://api.github.com/repos/huggingface/datasets/issues/comments/857038837/reactions
2021-06-08T19:22:32Z
https://api.github.com/repos/huggingface/datasets/issues/comments/857038837
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
## Describe the bug
load_from_disk and save_to_disk are not compatible. When I use save_to_disk to save a dataset to disk it works perfectly but given the same directory load_from_disk throws an error that it can't find state.json. looks like the load_from_disk only works on one split

## Steps to reproduce the bug
```python
from datasets import load_dataset
dataset = load_dataset("art")
dataset.save_to_disk("mydir")
d = Dataset.load_from_disk("mydir")
```

## Expected results
It is expected that these two functions be the reverse of each other without more manipulation

## Actual results
FileNotFoundError: [Errno 2] No such file or directory: 'mydir/art/state.json'

## Environment info
- `datasets` version: 1.6.2
- Platform: Linux-5.4.0-73-generic-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.10
- PyTorch version (GPU?): 1.8.1+cu102 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

