Add KS task to SUPERB
MEMBER
thanks a lot for implementing this @anton-l !!

i won't have time to review this while i'm away, so happy for @albertvillanova and @patrickvonplaten to decide when to merge :)
2021-08-11T14:47:51Z
https://github.com/huggingface/datasets/pull/2783#issuecomment-896891827
https://api.github.com/repos/huggingface/datasets/issues/2783
IC_kwDODunzps41dXuz
https://api.github.com/repos/huggingface/datasets/issues/comments/896891827/reactions
2021-08-11T14:47:51Z
https://api.github.com/repos/huggingface/datasets/issues/comments/896891827
https://avatars.githubusercontent.com/u/26859204?v=4
https://api.github.com/users/lewtun/events{/privacy}
https://api.github.com/users/lewtun/followers
https://api.github.com/users/lewtun/following{/other_user}
https://api.github.com/users/lewtun/gists{/gist_id}
https://github.com/lewtun
lewtun
MDQ6VXNlcjI2ODU5MjA0
https://api.github.com/users/lewtun/orgs
https://api.github.com/users/lewtun/received_events
https://api.github.com/users/lewtun/repos
https://api.github.com/users/lewtun/starred{/owner}{/repo}
https://api.github.com/users/lewtun/subscriptions
User
https://api.github.com/users/lewtun
public
MEMBER
@albertvillanova thanks! Everything should be ready now :)
2021-08-11T17:36:00Z
https://github.com/huggingface/datasets/pull/2783#issuecomment-897018229
https://api.github.com/repos/huggingface/datasets/issues/2783
IC_kwDODunzps41d2l1
https://api.github.com/repos/huggingface/datasets/issues/comments/897018229/reactions
2021-08-11T17:36:00Z
https://api.github.com/repos/huggingface/datasets/issues/comments/897018229
https://avatars.githubusercontent.com/u/26864830?v=4
https://api.github.com/users/anton-l/events{/privacy}
https://api.github.com/users/anton-l/followers
https://api.github.com/users/anton-l/following{/other_user}
https://api.github.com/users/anton-l/gists{/gist_id}
https://github.com/anton-l
anton-l
MDQ6VXNlcjI2ODY0ODMw
https://api.github.com/users/anton-l/orgs
https://api.github.com/users/anton-l/received_events
https://api.github.com/users/anton-l/repos
https://api.github.com/users/anton-l/starred{/owner}{/repo}
https://api.github.com/users/anton-l/subscriptions
User
https://api.github.com/users/anton-l
public
MEMBER
> The _background_noise_/_silence_ audio files are much longer than others, so they require some sort of slicing for downstream training. I decided to leave the implementation of that up to the users, since TFDS and s3prl take different approaches (either slicing wavs deterministically, or subsampling randomly at runtime)

@anton-l I was thinking that maybe we could give some hints in the dataset card (in a Usage section); something similar as for diarization: https://github.com/huggingface/datasets/blob/master/datasets/superb/README.md#example-of-usage
Note that for diarization it is not yet finished: we have to test it and then provide an end-to-end example: https://github.com/huggingface/datasets/pull/2661/files#r680224909 
2021-08-12T16:30:42Z
https://github.com/huggingface/datasets/pull/2783#issuecomment-897787071
https://api.github.com/repos/huggingface/datasets/issues/2783
IC_kwDODunzps41gyS_
https://api.github.com/repos/huggingface/datasets/issues/comments/897787071/reactions
2021-08-12T16:30:42Z
https://api.github.com/repos/huggingface/datasets/issues/comments/897787071
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
MEMBER
@albertvillanova yeah, I'm not sure how to best implement it in pure `datasets` yet. It's something like this, where `sample_noise()` needs to be called from a pytorch batch collator or other framework-specific variant:

```python
def map_to_array(example):
    import soundfile as sf

    speech_array, sample_rate = sf.read(example["file"])
    example["speech"] = speech_array
    example["sample_rate"] = sample_rate
    return example


def sample_noise(example):
    # Use a version of this function in a stateless way to extract random 1 sec slices of background noise
    # on each epoch
    from random import randint

    # _silence_ audios are longer than 1 sec
    if example["label"] == "_silence_":
        random_offset = randint(0, len(example["speech"]) - example["sample_rate"] - 1)
        example["speech"] = example["speech"][random_offset : random_offset + example["sample_rate"]]

    return example
```
2021-08-12T16:40:26Z
https://github.com/huggingface/datasets/pull/2783#issuecomment-897793812
https://api.github.com/repos/huggingface/datasets/issues/2783
IC_kwDODunzps41gz8U
https://api.github.com/repos/huggingface/datasets/issues/comments/897793812/reactions
2021-08-12T16:40:26Z
https://api.github.com/repos/huggingface/datasets/issues/comments/897793812
https://avatars.githubusercontent.com/u/26864830?v=4
https://api.github.com/users/anton-l/events{/privacy}
https://api.github.com/users/anton-l/followers
https://api.github.com/users/anton-l/following{/other_user}
https://api.github.com/users/anton-l/gists{/gist_id}
https://github.com/anton-l
anton-l
MDQ6VXNlcjI2ODY0ODMw
https://api.github.com/users/anton-l/orgs
https://api.github.com/users/anton-l/received_events
https://api.github.com/users/anton-l/repos
https://api.github.com/users/anton-l/starred{/owner}{/repo}
https://api.github.com/users/anton-l/subscriptions
User
https://api.github.com/users/anton-l
public
MEMBER
I see... Yes, not trivial indeed. Maybe for the moment you could add those functions above to the README (as it is the case for now in diarization)? What do you think?
2021-08-12T16:45:00Z
https://github.com/huggingface/datasets/pull/2783#issuecomment-897796762
https://api.github.com/repos/huggingface/datasets/issues/2783
IC_kwDODunzps41g0qa
https://api.github.com/repos/huggingface/datasets/issues/comments/897796762/reactions
2021-08-12T16:45:00Z
https://api.github.com/repos/huggingface/datasets/issues/comments/897796762
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
Add the KS (keyword spotting) task as described in the [SUPERB paper](https://arxiv.org/abs/2105.01051).

- [s3prl instructions](https://github.com/s3prl/s3prl/blob/master/s3prl/downstream/README.md#ks-keyword-spotting)
- [s3prl implementation](https://github.com/s3prl/s3prl/blob/master/s3prl/downstream/speech_commands/dataset.py)
- [TFDS implementation](https://github.com/tensorflow/datasets/blob/master/tensorflow_datasets/audio/speech_commands.py)

Some notable quirks:
- The dataset is originally single-archive (train+val+test all in one), but the test set has a "canonical" distribution in a separate archive, which is also used here (see `_split_ks_files()`). 
- The `_background_noise_`/`_silence_` audio files are much longer than others, so they require some sort of slicing for downstream training. I decided to leave the implementation of that up to the users, since TFDS and s3prl take different approaches (either slicing wavs deterministically, or subsampling randomly at runtime)

Related to #2619.