Support Audio feature for TAR archives in sequential access
MEMBER
Also do you think we can adapt `cast_column` to keep the same value for this new parameter when the user only wants to change the sampling rate ?
2021-10-25T09:42:43Z
https://github.com/huggingface/datasets/pull/3129#issuecomment-950730528
https://api.github.com/repos/huggingface/datasets/issues/3129
IC_kwDODunzps44qv8g
https://api.github.com/repos/huggingface/datasets/issues/comments/950730528/reactions
2021-10-25T09:42:43Z
https://api.github.com/repos/huggingface/datasets/issues/comments/950730528
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
Thanks for your comments, @lhoestq, I will address them afterwards.

But, I think it is more important/urgent first address the current blocking non-passing test: https://github.com/huggingface/datasets/runs/4143579241?check_suite_focus=true
- I am thinking of a way of solving it, but if you have any hint, it will be more than welcome! 😅 

Basically:
```
{'audio': '/tmp/pytest-of-runner/pytest-0/popen-gw1/test_dataset_with_audio_featur1/data/test_audio_44100.wav'}
``` 
becomes
```
{'audio': {'bytes': None, 'path': '/tmp/pytest-of-runner/pytest-0/popen-gw1/test_dataset_with_audio_featur1/data/test_audio_44100.wav'}}
```
after a `map`, which is what was stored in the Arrow file. However we expect it remains invariant after this `map`.
2021-11-09T17:00:40Z
https://github.com/huggingface/datasets/pull/3129#issuecomment-964347056
https://api.github.com/repos/huggingface/datasets/issues/3129
IC_kwDODunzps45esSw
https://api.github.com/repos/huggingface/datasets/issues/comments/964347056/reactions
2021-11-10T14:35:48Z
https://api.github.com/repos/huggingface/datasets/issues/comments/964347056
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
MEMBER
@lhoestq, @mariosasko I finally proposed another implementation different from my last one:
- Before: store Audio always a struct<path: string, bytes: binary>, where bytes can be None
- Now, depending on the examples, either store Audio as a struct (as before), or as a string.

Please note that the main motivation for this change was the issue mentioned above: https://github.com/huggingface/datasets/pull/3129#issuecomment-964347056

2021-11-10T14:28:11Z
https://github.com/huggingface/datasets/pull/3129#issuecomment-965269123
https://api.github.com/repos/huggingface/datasets/issues/3129
IC_kwDODunzps45iNaD
https://api.github.com/repos/huggingface/datasets/issues/comments/965269123/reactions
2021-11-10T14:30:29Z
https://api.github.com/repos/huggingface/datasets/issues/comments/965269123
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
MEMBER
Until here we had the assumption that a Features object always has an associated, deterministic, pyarrow schema. This is useful to ensure that we are able to concatenate two datasets that have the same features for example.

By breaking this assumption for the Audio type, how can we ensure that we can concatenate two audio datasets if one has Audio as a struct and the other a string ?
2021-11-10T14:49:50Z
https://github.com/huggingface/datasets/pull/3129#issuecomment-965314621
https://api.github.com/repos/huggingface/datasets/issues/3129
IC_kwDODunzps45iYg9
https://api.github.com/repos/huggingface/datasets/issues/comments/965314621/reactions
2021-11-10T14:49:50Z
https://api.github.com/repos/huggingface/datasets/issues/comments/965314621
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
Oh I noticed that the Audio feature type has a private attribute `_storage_dtype`, so the assumption still holds, since they are now different feature types depending on the this attribute :)
(i mean different from the python equal operator point of view)
2021-11-10T15:05:19Z
https://github.com/huggingface/datasets/pull/3129#issuecomment-965347957
https://api.github.com/repos/huggingface/datasets/issues/3129
IC_kwDODunzps45igp1
https://api.github.com/repos/huggingface/datasets/issues/comments/965347957/reactions
2021-11-10T15:07:04Z
https://api.github.com/repos/huggingface/datasets/issues/comments/965347957
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
I think this PR is ready, @lhoestq, @mariosasko.  
2021-11-16T14:58:09Z
https://github.com/huggingface/datasets/pull/3129#issuecomment-970355651
https://api.github.com/repos/huggingface/datasets/issues/3129
IC_kwDODunzps451nPD
https://api.github.com/repos/huggingface/datasets/issues/comments/970355651/reactions
2021-11-16T14:58:09Z
https://api.github.com/repos/huggingface/datasets/issues/comments/970355651
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
COLLABORATOR
Nit: We should also mention the new storage structure in the `Features` docstring [here](https://github.com/huggingface/datasets/blob/b29fb550c31de337b952035a7584147e0f18c0cf/src/datasets/features/features.py#L966) for users to know what type of value to return in their dataset scripts (we also have a link to that docstring in the `ADD_NEW_DATASET` template).
2021-11-17T15:17:25Z
https://github.com/huggingface/datasets/pull/3129#issuecomment-971680358
https://api.github.com/repos/huggingface/datasets/issues/3129
IC_kwDODunzps456qpm
https://api.github.com/repos/huggingface/datasets/issues/comments/971680358/reactions
2021-11-17T15:17:25Z
https://api.github.com/repos/huggingface/datasets/issues/comments/971680358
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
Add Audio feature support for TAR archived files in sequential access.

Fix #3128.