Add code examples to API docs
NONE
_The documentation is not available anymore as the PR was closed or merged._
2022-04-13T23:14:00Z
https://github.com/huggingface/datasets/pull/4168#issuecomment-1098564527
https://api.github.com/repos/huggingface/datasets/issues/4168
IC_kwDODunzps5BesOv
https://api.github.com/repos/huggingface/datasets/issues/comments/1098564527/reactions
2022-04-27T18:53:37Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1098564527
https://avatars.githubusercontent.com/u/99929124?v=4
https://api.github.com/users/HuggingFaceDocBuilderDev/events{/privacy}
https://api.github.com/users/HuggingFaceDocBuilderDev/followers
https://api.github.com/users/HuggingFaceDocBuilderDev/following{/other_user}
https://api.github.com/users/HuggingFaceDocBuilderDev/gists{/gist_id}
https://github.com/HuggingFaceDocBuilderDev
HuggingFaceDocBuilderDev
U_kgDOBfTMJA
https://api.github.com/users/HuggingFaceDocBuilderDev/orgs
https://api.github.com/users/HuggingFaceDocBuilderDev/received_events
https://api.github.com/users/HuggingFaceDocBuilderDev/repos
https://api.github.com/users/HuggingFaceDocBuilderDev/starred{/owner}{/repo}
https://api.github.com/users/HuggingFaceDocBuilderDev/subscriptions
User
https://api.github.com/users/HuggingFaceDocBuilderDev
public
MEMBER
> Do you think it is clearer to make every code example fully reproducible so when users copy the code they can actually run it and get an output? This seems quite repetitive - maybe even unnecessary - but it is definitely clearer.

I think it's ok to be repetitive to get more clarity. Many users come from `transformers` and may have little experience with some processing methods (especially torch users).

> Should we showcase a function with more than one parameter to highlight different use-cases (it's pretty basic right now, but I'd be happy to add more)?

Maybe let's do it case by case, depending on whether there are parameters that are likely to be used often ?

> For the class_encode_column function, let me know if there is a simpler dataset with fewer columns (currently using winograd_wsc) so it is easier for users to see what changed.

You can try with `boolq`, it has a boolean column that can be converted to labels

> Where possible, I try to show the input before and the output after using a function like flatten for example. Do you think this is too much and just showing the usage (ie, >>> ds.flatten()) will be sufficient?

No I don't think it's too much, it's nice this way thanks :)
2022-04-14T12:09:49Z
https://github.com/huggingface/datasets/pull/4168#issuecomment-1099119367
https://api.github.com/repos/huggingface/datasets/issues/4168
IC_kwDODunzps5BgzsH
https://api.github.com/repos/huggingface/datasets/issues/comments/1099119367/reactions
2022-04-14T12:09:49Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1099119367
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
Updated each code example so they are fully reproducible (where applicable)! The next step will be to identify some functions where we can show off some parameters that are useful or commonly used. Some useful parameters can be:

- use `map(batched=True)` to process batches of examples.
- set a seed in `shuffle`.
- set `shuffle` and `seed` in `train_test_split`.

Let me know if you think of anything else related to the functions in `arrow_dataset.py`!
2022-04-14T19:47:59Z
https://github.com/huggingface/datasets/pull/4168#issuecomment-1099566681
https://api.github.com/repos/huggingface/datasets/issues/4168
IC_kwDODunzps5Big5Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1099566681/reactions
2022-04-14T19:47:59Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1099566681
https://avatars.githubusercontent.com/u/59462357?v=4
https://api.github.com/users/stevhliu/events{/privacy}
https://api.github.com/users/stevhliu/followers
https://api.github.com/users/stevhliu/following{/other_user}
https://api.github.com/users/stevhliu/gists{/gist_id}
https://github.com/stevhliu
stevhliu
MDQ6VXNlcjU5NDYyMzU3
https://api.github.com/users/stevhliu/orgs
https://api.github.com/users/stevhliu/received_events
https://api.github.com/users/stevhliu/repos
https://api.github.com/users/stevhliu/starred{/owner}{/repo}
https://api.github.com/users/stevhliu/subscriptions
User
https://api.github.com/users/stevhliu
public
MEMBER
Cool thanks ! I think you can also do `num_proc` for `map`
2022-04-15T15:25:40Z
https://github.com/huggingface/datasets/pull/4168#issuecomment-1100175054
https://api.github.com/repos/huggingface/datasets/issues/4168
IC_kwDODunzps5Bk1bO
https://api.github.com/repos/huggingface/datasets/issues/comments/1100175054/reactions
2022-04-15T15:25:40Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1100175054
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
This PR adds code examples for functions related to the base Datasets class to highlight usage. Most of the examples use the `rotten_tomatoes` dataset since it is nice and small. Several things I would appreciate feedback on:

- Do you think it is clearer to make every code example fully reproducible so when users copy the code they can actually run it and get an output? This seems quite repetitive - maybe even unnecessary - but it is definitely clearer. Personally, I think we might be able to get away with not including this since users probably want to try the function on their own dataset. For example:

   ```py
   >>> from datasets import load_dataset
   >>> ds = load_dataset("rotten_tomatoes", split="validation")
   >>> code example goes here
   ```

- Should we showcase a function with more than one parameter to highlight different use-cases (it's pretty basic right now, but I'd be happy to add more)?
- For the `class_encode_column` function, let me know if there is a simpler dataset with fewer columns (currently using `winograd_wsc`) so it is easier for users to see what changed.
- Where possible, I try to show the input before and the output after using a function like `flatten` for example. Do you think this is too much and just showing the usage (ie, `>>> ds.flatten()`) will be sufficient?

Thanks :)