[save_to_disk] Using `select()` followed by `save_to_disk` saves complete dataset making it hard to create dummy dataset
COLLABORATOR
Hi,

the `save_to_disk` docstring explains that `flatten_indices` has to be called on a dataset before saving it to save only the shard/slice of the dataset.
2021-10-06T22:05:02Z
https://github.com/huggingface/datasets/issues/3040#issuecomment-937231649
https://api.github.com/repos/huggingface/datasets/issues/3040
IC_kwDODunzps433QUh
https://api.github.com/repos/huggingface/datasets/issues/comments/937231649/reactions
2021-10-06T22:05:02Z
https://api.github.com/repos/huggingface/datasets/issues/comments/937231649
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
CONTRIBUTOR
That works! Thansk!

Might be worth doing that automatically actually in case the `save_to_disk` is called on a dataset that has an indices mapping :-)
2021-10-06T23:33:33Z
https://github.com/huggingface/datasets/issues/3040#issuecomment-937323939
https://api.github.com/repos/huggingface/datasets/issues/3040
IC_kwDODunzps433m2j
https://api.github.com/repos/huggingface/datasets/issues/comments/937323939/reactions
2021-10-06T23:33:33Z
https://api.github.com/repos/huggingface/datasets/issues/comments/937323939
https://avatars.githubusercontent.com/u/23423619?v=4
https://api.github.com/users/patrickvonplaten/events{/privacy}
https://api.github.com/users/patrickvonplaten/followers
https://api.github.com/users/patrickvonplaten/following{/other_user}
https://api.github.com/users/patrickvonplaten/gists{/gist_id}
https://github.com/patrickvonplaten
patrickvonplaten
MDQ6VXNlcjIzNDIzNjE5
https://api.github.com/users/patrickvonplaten/orgs
https://api.github.com/users/patrickvonplaten/received_events
https://api.github.com/users/patrickvonplaten/repos
https://api.github.com/users/patrickvonplaten/starred{/owner}{/repo}
https://api.github.com/users/patrickvonplaten/subscriptions
User
https://api.github.com/users/patrickvonplaten
public
MEMBER
I agree with @patrickvonplaten: this issue is reported recurrently, so better if we implement the `.flatten_indices()` automatically?
2021-10-07T06:59:00Z
https://github.com/huggingface/datasets/issues/3040#issuecomment-937509520
https://api.github.com/repos/huggingface/datasets/issues/3040
IC_kwDODunzps434UKQ
https://api.github.com/repos/huggingface/datasets/issues/comments/937509520/reactions
2021-10-07T06:59:00Z
https://api.github.com/repos/huggingface/datasets/issues/comments/937509520
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
CONTRIBUTOR
That would be great indeed - I don't really see a use case where one would not like to call `.flatten_indices()` before calling `save_to_disk`
2021-10-07T09:37:09Z
https://github.com/huggingface/datasets/issues/3040#issuecomment-937623273
https://api.github.com/repos/huggingface/datasets/issues/3040
IC_kwDODunzps434v7p
https://api.github.com/repos/huggingface/datasets/issues/comments/937623273/reactions
2021-10-07T09:37:09Z
https://api.github.com/repos/huggingface/datasets/issues/comments/937623273
https://avatars.githubusercontent.com/u/23423619?v=4
https://api.github.com/users/patrickvonplaten/events{/privacy}
https://api.github.com/users/patrickvonplaten/followers
https://api.github.com/users/patrickvonplaten/following{/other_user}
https://api.github.com/users/patrickvonplaten/gists{/gist_id}
https://github.com/patrickvonplaten
patrickvonplaten
MDQ6VXNlcjIzNDIzNjE5
https://api.github.com/users/patrickvonplaten/orgs
https://api.github.com/users/patrickvonplaten/received_events
https://api.github.com/users/patrickvonplaten/repos
https://api.github.com/users/patrickvonplaten/starred{/owner}{/repo}
https://api.github.com/users/patrickvonplaten/subscriptions
User
https://api.github.com/users/patrickvonplaten
public
MEMBER
+1 on this !
2021-10-07T12:37:35Z
https://github.com/huggingface/datasets/issues/3040#issuecomment-937751233
https://api.github.com/repos/huggingface/datasets/issues/3040
IC_kwDODunzps435PLB
https://api.github.com/repos/huggingface/datasets/issues/comments/937751233/reactions
2021-10-07T12:37:35Z
https://api.github.com/repos/huggingface/datasets/issues/comments/937751233
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
## Describe the bug

When only keeping a dummy size of a dataset (say the first 100 samples), and then saving it  to disk to upload it in the following to the hub for easy demo/use - not just the small dataset is saved but the whole dataset with an indices file. The problem with this is that the dataset is still very big.

## Steps to reproduce the bug

E.g. run the following:

```python
from datasets import load_dataset, save_to_disk

nlp = load_dataset("glue", "mnli", split="train")
nlp.save_to_disk("full")

nlp = nlp.select(range(100))
nlp.save_to_disk("dummy")
```

Now one can see that both `"dummy"` and `"full"` have the same size. This shouldn't be the case IMO.

## Expected results

IMO `"dummy"` should be much smaller so that one can easily play around with the dataset on the hub.

## Actual results
Specify the actual results or traceback.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->

- `datasets` version: 1.12.2.dev0
- Platform: Linux-5.11.0-34-generic-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyArrow version: 5.0.0

