why the stratify option is omitted from test_train_split function?
MEMBER
Hi ! It's simply not added yet :)

If someone wants to contribute to add the `stratify` parameter I'd be happy to give some pointers.

In the meantime, I guess you can use `sklearn` or other tools to do a stratified train/test split over the **indices** of your dataset and then do
```
train_dataset = dataset.select(train_indices)
test_dataset = dataset.select(test_indices)
```
2022-01-05T15:57:56Z
https://github.com/huggingface/datasets/issues/3452#issuecomment-1005851500
https://api.github.com/repos/huggingface/datasets/issues/3452
IC_kwDODunzps479BNs
https://api.github.com/repos/huggingface/datasets/issues/comments/1005851500/reactions
2022-01-05T15:57:56Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1005851500
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Hi @lhoestq I would like to add `stratify` parameter, can you give me some pointers for adding the same ?
2022-05-05T04:52:25Z
https://github.com/huggingface/datasets/issues/3452#issuecomment-1118171985
https://api.github.com/repos/huggingface/datasets/issues/3452
IC_kwDODunzps5CpfNR
https://api.github.com/repos/huggingface/datasets/issues/comments/1118171985/reactions
2022-05-05T04:53:37Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1118171985
https://avatars.githubusercontent.com/u/48522685?v=4
https://api.github.com/users/nandwalritik/events{/privacy}
https://api.github.com/users/nandwalritik/followers
https://api.github.com/users/nandwalritik/following{/other_user}
https://api.github.com/users/nandwalritik/gists{/gist_id}
https://github.com/nandwalritik
nandwalritik
MDQ6VXNlcjQ4NTIyNjg1
https://api.github.com/users/nandwalritik/orgs
https://api.github.com/users/nandwalritik/received_events
https://api.github.com/users/nandwalritik/repos
https://api.github.com/users/nandwalritik/starred{/owner}{/repo}
https://api.github.com/users/nandwalritik/subscriptions
User
https://api.github.com/users/nandwalritik
public
MEMBER
Hi ! Sure :)

The `train_test_split` method is defined here: 

https://github.com/huggingface/datasets/blob/dc62232fa1b3bcfe2fbddcb721f2d141f8908943/src/datasets/arrow_dataset.py#L3253-L3253

and inside `train_test_split ` we need to create the right `train_indices` and `test_indices` that are passed here to `.select()`:

https://github.com/huggingface/datasets/blob/dc62232fa1b3bcfe2fbddcb721f2d141f8908943/src/datasets/arrow_dataset.py#L3450-L3464

For example if your dataset is like
|    |   label |
|---:|--------:|
|  0 |       1 |
|  1 |       1 |
|  2 |       0 |
|  3 |       0 |

and the user passes `stratify=dataset["label"]`, then you should get indices that look like this
```
train_indices = [0, 2]
test_indices = [1, 3]
```

these indices will be passed to `.select` to return the stratified train and test splits :)

Feel free to îng me if you have any question !
2022-05-05T13:07:51Z
https://github.com/huggingface/datasets/issues/3452#issuecomment-1118527060
https://api.github.com/repos/huggingface/datasets/issues/3452
IC_kwDODunzps5Cq15U
https://api.github.com/repos/huggingface/datasets/issues/comments/1118527060/reactions
2022-05-05T13:07:51Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1118527060
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
@lhoestq 
I just added the implementation for `stratify` option here #4322 
2022-05-12T08:02:54Z
https://github.com/huggingface/datasets/issues/3452#issuecomment-1124659760
https://api.github.com/repos/huggingface/datasets/issues/3452
IC_kwDODunzps5DCPIw
https://api.github.com/repos/huggingface/datasets/issues/comments/1124659760/reactions
2022-05-12T08:02:54Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1124659760
https://avatars.githubusercontent.com/u/48522685?v=4
https://api.github.com/users/nandwalritik/events{/privacy}
https://api.github.com/users/nandwalritik/followers
https://api.github.com/users/nandwalritik/following{/other_user}
https://api.github.com/users/nandwalritik/gists{/gist_id}
https://github.com/nandwalritik
nandwalritik
MDQ6VXNlcjQ4NTIyNjg1
https://api.github.com/users/nandwalritik/orgs
https://api.github.com/users/nandwalritik/received_events
https://api.github.com/users/nandwalritik/repos
https://api.github.com/users/nandwalritik/starred{/owner}{/repo}
https://api.github.com/users/nandwalritik/subscriptions
User
https://api.github.com/users/nandwalritik
public
why the stratify option is omitted from test_train_split function?

is there any other way implement the stratify option while splitting the dataset? as it is important point to be considered while splitting the dataset.