Image & Audio formatting for numpy/torch/tf/jax
NONE
_The documentation is not available anymore as the PR was closed or merged._
2022-10-05T13:11:21Z
https://github.com/huggingface/datasets/pull/5072#issuecomment-1268416622
https://api.github.com/repos/huggingface/datasets/issues/5072
IC_kwDODunzps5LmoBu
https://api.github.com/repos/huggingface/datasets/issues/comments/1268416622/reactions
2022-10-10T13:24:09Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1268416622
https://avatars.githubusercontent.com/u/99929124?v=4
https://api.github.com/users/HuggingFaceDocBuilderDev/events{/privacy}
https://api.github.com/users/HuggingFaceDocBuilderDev/followers
https://api.github.com/users/HuggingFaceDocBuilderDev/following{/other_user}
https://api.github.com/users/HuggingFaceDocBuilderDev/gists{/gist_id}
https://github.com/HuggingFaceDocBuilderDev
HuggingFaceDocBuilderDev
U_kgDOBfTMJA
https://api.github.com/users/HuggingFaceDocBuilderDev/orgs
https://api.github.com/users/HuggingFaceDocBuilderDev/received_events
https://api.github.com/users/HuggingFaceDocBuilderDev/repos
https://api.github.com/users/HuggingFaceDocBuilderDev/starred{/owner}{/repo}
https://api.github.com/users/HuggingFaceDocBuilderDev/subscriptions
User
https://api.github.com/users/HuggingFaceDocBuilderDev
public
MEMBER
I just added a consolidation step so that numpy arrays or tensors of images are stacked together if the shapes match, instead of having lists of tensors

Feel free to review @mariosasko :)
2022-10-06T14:19:34Z
https://github.com/huggingface/datasets/pull/5072#issuecomment-1270140471
https://api.github.com/repos/huggingface/datasets/issues/5072
IC_kwDODunzps5LtM43
https://api.github.com/repos/huggingface/datasets/issues/comments/1270140471/reactions
2022-10-06T14:20:39Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1270140471
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
I added a few lines in the docs and reverted the ragged numpy array change :)

ready for another review @mariosasko !
2022-10-07T18:25:13Z
https://github.com/huggingface/datasets/pull/5072#issuecomment-1271923882
https://api.github.com/repos/huggingface/datasets/issues/5072
IC_kwDODunzps5L0ASq
https://api.github.com/repos/huggingface/datasets/issues/comments/1271923882/reactions
2022-10-07T18:25:24Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1271923882
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
Added support for image and audio formatting for numpy, torch, tf and jax.

For images, the dtype used is the one of the image (the one returned by PIL.Image), e.g. uint8

I also added support for string, binary and None types. In particular for torch and jax, strings are kept unchanged (previously it was returning an error because you can't create a tensor of strings)