Length of table does not accurately reflect the split
MEMBER
As already replied by @lhoestq (private channel):
> `.train_test_split` (as well as `.shard`, `.select`) doesn't create a new arrow table to save time and disk space. Instead, it uses an indices mapping on top of the table that locate which examples are part of train or test.
2023-06-02T11:57:14Z
https://github.com/huggingface/datasets/issues/5922#issuecomment-1573616652
https://api.github.com/repos/huggingface/datasets/issues/5922
IC_kwDODunzps5dy3wM
https://api.github.com/repos/huggingface/datasets/issues/comments/1573616652/reactions
2023-06-02T11:57:14Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1573616652
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
COLLABORATOR
This is an optimization that we don't plan to "fix", so I'm closing this issue.
2023-06-02T16:13:27Z
https://github.com/huggingface/datasets/issues/5922#issuecomment-1573984625
https://api.github.com/repos/huggingface/datasets/issues/5922
IC_kwDODunzps5d0Rlx
https://api.github.com/repos/huggingface/datasets/issues/comments/1573984625/reactions
2023-06-02T16:13:27Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1573984625
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
### Describe the bug

I load a Huggingface Dataset and do `train_test_split`. I'm expecting the underlying table for the dataset to also be split, but it's not.

### Steps to reproduce the bug

![image](https://github.com/huggingface/datasets/assets/8068268/83e5768f-8b4c-422a-945c-832a7585afff)


### Expected behavior

The expected behavior is when `len(hf_dataset["train"].data)` should match the length of the train split, and not be the entire unsplit dataset.

### Environment info

datasets 2.10.1
python 3.10.11