Support saving datasets with custom formatting
MEMBER
Not a fan of pickling this sort of stuff either.
Note that users can also share the code in their dataset documentation.
2023-09-06T18:32:07Z
https://github.com/huggingface/datasets/issues/6221#issuecomment-1708893059
https://api.github.com/repos/huggingface/datasets/issues/6221
IC_kwDODunzps5l26OD
https://api.github.com/repos/huggingface/datasets/issues/comments/1708893059/reactions
2023-09-06T18:32:07Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1708893059
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
Requested in https://discuss.huggingface.co/t/using-set-transform-on-a-dataset-leads-to-an-exception/53036.

I am not sure if supporting this is the best idea for the following reasons:

>For this to work, we would have to pickle a custom transform, which means the transform and the objects it references need to be serializable. Also, deserializing these bytes would make `load_from_disk` unsafe, so I'm not sure this is a good idea. 

@lhoestq WDYT?
