Manipulate columns on IterableDataset (rename columns, cast, etc.)
NONE
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_3862). All of your documentation changes will be reflected on that endpoint.
2022-03-08T14:59:10Z
https://github.com/huggingface/datasets/pull/3862#issuecomment-1061868762
https://api.github.com/repos/huggingface/datasets/issues/3862
IC_kwDODunzps4_StTa
https://api.github.com/repos/huggingface/datasets/issues/comments/1061868762/reactions
2022-03-08T14:59:10Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1061868762
https://avatars.githubusercontent.com/u/99929124?v=4
https://api.github.com/users/HuggingFaceDocBuilderDev/events{/privacy}
https://api.github.com/users/HuggingFaceDocBuilderDev/followers
https://api.github.com/users/HuggingFaceDocBuilderDev/following{/other_user}
https://api.github.com/users/HuggingFaceDocBuilderDev/gists{/gist_id}
https://github.com/HuggingFaceDocBuilderDev
HuggingFaceDocBuilderDev
U_kgDOBfTMJA
https://api.github.com/users/HuggingFaceDocBuilderDev/orgs
https://api.github.com/users/HuggingFaceDocBuilderDev/received_events
https://api.github.com/users/HuggingFaceDocBuilderDev/repos
https://api.github.com/users/HuggingFaceDocBuilderDev/starred{/owner}{/repo}
https://api.github.com/users/HuggingFaceDocBuilderDev/subscriptions
User
https://api.github.com/users/HuggingFaceDocBuilderDev
public
MEMBER
> IIUC we check if columns are present/not present directly in the yielded examples and not in info.features because info.features can be None (after map, for instance)?

Yes exactly

> We should develop a solution that ensures info.features is never None. For example, one approach would be to infer them from examples in map and make them promotable from Value("null") to a specific type, in case of None values.

I agree this would be useful. Though inferring the type requires to start streaming some data, which takes a few seconds (compared to being instantaneous right now).

Let's discuss this in a new issue maybe ?
2022-03-10T14:55:18Z
https://github.com/huggingface/datasets/pull/3862#issuecomment-1064149424
https://api.github.com/repos/huggingface/datasets/issues/3862
IC_kwDODunzps4_baGw
https://api.github.com/repos/huggingface/datasets/issues/comments/1064149424/reactions
2022-03-10T14:55:18Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1064149424
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
I added:
- add_column
- cast
- rename_column
- rename_columns

related to https://github.com/huggingface/datasets/issues/3444

TODO:
- [x] docs
- [x] tests