add DatasetDict
MEMBER
I did the changes in the docstrings and I added a type check in each `DatasetDict` method to make sure all values are of type `Dataset`
2020-07-24T08:21:30Z
https://github.com/huggingface/datasets/pull/430#issuecomment-663405485
https://api.github.com/repos/huggingface/datasets/issues/430
MDEyOklzc3VlQ29tbWVudDY2MzQwNTQ4NQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/663405485/reactions
2020-07-24T08:21:30Z
https://api.github.com/repos/huggingface/datasets/issues/comments/663405485
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
Awesome, do you mind adding these in the doc as well?
2020-07-25T09:06:10Z
https://github.com/huggingface/datasets/pull/430#issuecomment-663831891
https://api.github.com/repos/huggingface/datasets/issues/430
MDEyOklzc3VlQ29tbWVudDY2MzgzMTg5MQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/663831891/reactions
2020-07-25T09:06:10Z
https://api.github.com/repos/huggingface/datasets/issues/comments/663831891
https://avatars.githubusercontent.com/u/7353373?v=4
https://api.github.com/users/thomwolf/events{/privacy}
https://api.github.com/users/thomwolf/followers
https://api.github.com/users/thomwolf/following{/other_user}
https://api.github.com/users/thomwolf/gists{/gist_id}
https://github.com/thomwolf
thomwolf
MDQ6VXNlcjczNTMzNzM=
https://api.github.com/users/thomwolf/orgs
https://api.github.com/users/thomwolf/received_events
https://api.github.com/users/thomwolf/repos
https://api.github.com/users/thomwolf/starred{/owner}{/repo}
https://api.github.com/users/thomwolf/subscriptions
User
https://api.github.com/users/thomwolf
public
MEMBER
I added it to the docs (processing + main classes)
2020-07-27T10:32:18Z
https://github.com/huggingface/datasets/pull/430#issuecomment-664288522
https://api.github.com/repos/huggingface/datasets/issues/430
MDEyOklzc3VlQ29tbWVudDY2NDI4ODUyMg==
https://api.github.com/repos/huggingface/datasets/issues/comments/664288522/reactions
2020-07-27T10:32:18Z
https://api.github.com/repos/huggingface/datasets/issues/comments/664288522
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
I'm trying to follow along with the following about datasets from the docs:

https://huggingface.co/nlp/loading_datasets.html
https://huggingface.co/nlp/processing.html

However the train_test_split method no longer works as it is expecting a dataset, rather than a datsetdict. How would I got about splitting a CSV into a train and test set? 

I'm trying to utilize the Trainer() class, but am having trouble converting my data from a csv into dataset objects to pass in.
2020-08-04T01:01:53Z
https://github.com/huggingface/datasets/pull/430#issuecomment-668315111
https://api.github.com/repos/huggingface/datasets/issues/430
MDEyOklzc3VlQ29tbWVudDY2ODMxNTExMQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/668315111/reactions
2020-08-04T01:01:53Z
https://api.github.com/repos/huggingface/datasets/issues/comments/668315111
https://avatars.githubusercontent.com/u/67607180?v=4
https://api.github.com/users/HugToDebug/events{/privacy}
https://api.github.com/users/HugToDebug/followers
https://api.github.com/users/HugToDebug/following{/other_user}
https://api.github.com/users/HugToDebug/gists{/gist_id}
https://github.com/HugToDebug
HugToDebug
MDQ6VXNlcjY3NjA3MTgw
https://api.github.com/users/HugToDebug/orgs
https://api.github.com/users/HugToDebug/received_events
https://api.github.com/users/HugToDebug/repos
https://api.github.com/users/HugToDebug/starred{/owner}{/repo}
https://api.github.com/users/HugToDebug/subscriptions
User
https://api.github.com/users/HugToDebug
public
## Add DatasetDict

### Overview

When you call `load_dataset` it can return a dictionary of datasets if there are several splits (train/test for example).
If you wanted to apply dataset transforms you had to iterate over each split and apply the transform.

Instead of returning a dict, it now returns a `nlp.DatasetDict` object which inherits from dict and contains the same data as before, except that now users can call dataset transforms directly from the output, and they'll be applied on each split.

Before:
```python
from nlp import load_dataset

squad = load_dataset("squad")
print(squad.keys())
# dict_keys(['train', 'validation'])
squad = {
    split_name: dataset.map(my_func) for split_name, dataset in squad.items()
}
print(squad.keys())
# dict_keys(['train', 'validation'])
```

Now:
```python
from nlp import load_dataset

squad = load_dataset("squad")
print(squad.keys())
# dict_keys(['train', 'validation'])
squad = squad.map(my_func)
print(squad.keys())
# dict_keys(['train', 'validation'])
```

### Dataset transforms

`nlp.DatasetDict` implements the following dataset transforms:
- map
- filter
- sort
- shuffle

### Arguments

The arguments of the methods are the same except for split-specific arguments like `cache_file_name`.
For such arguments, the expected input is a dictionary `{split_name: argument_value}`
It concerns:
- `cache_file_name` in map, filter, sort, shuffle
- `seed` and `generator` in shuffle