Add ASR task for SUPERB
MEMBER
Wait until #2620 is merged before pushing the README tags in this PR
2021-07-12T12:28:06Z
https://github.com/huggingface/datasets/pull/2619#issuecomment-878238274
https://api.github.com/repos/huggingface/datasets/issues/2619
MDEyOklzc3VlQ29tbWVudDg3ODIzODI3NA==
https://api.github.com/repos/huggingface/datasets/issues/comments/878238274/reactions
2021-07-12T12:28:06Z
https://api.github.com/repos/huggingface/datasets/issues/comments/878238274
https://avatars.githubusercontent.com/u/26859204?v=4
https://api.github.com/users/lewtun/events{/privacy}
https://api.github.com/users/lewtun/followers
https://api.github.com/users/lewtun/following{/other_user}
https://api.github.com/users/lewtun/gists{/gist_id}
https://github.com/lewtun
lewtun
MDQ6VXNlcjI2ODU5MjA0
https://api.github.com/users/lewtun/orgs
https://api.github.com/users/lewtun/received_events
https://api.github.com/users/lewtun/repos
https://api.github.com/users/lewtun/starred{/owner}{/repo}
https://api.github.com/users/lewtun/subscriptions
User
https://api.github.com/users/lewtun
public
MEMBER
> Thanks!
> 
> One question: aren't you adding `task_templates` to the `_info` method (and to the `dataset_infos.json`?

great catch! i've now added the asr task template (along with a mapping from superb task -> template) and updated the `dataset_infos.json` :) 
2021-07-13T06:28:40Z
https://github.com/huggingface/datasets/pull/2619#issuecomment-878815706
https://api.github.com/repos/huggingface/datasets/issues/2619
MDEyOklzc3VlQ29tbWVudDg3ODgxNTcwNg==
https://api.github.com/repos/huggingface/datasets/issues/comments/878815706/reactions
2021-07-13T06:28:40Z
https://api.github.com/repos/huggingface/datasets/issues/comments/878815706
https://avatars.githubusercontent.com/u/26859204?v=4
https://api.github.com/users/lewtun/events{/privacy}
https://api.github.com/users/lewtun/followers
https://api.github.com/users/lewtun/following{/other_user}
https://api.github.com/users/lewtun/gists{/gist_id}
https://github.com/lewtun
lewtun
MDQ6VXNlcjI2ODU5MjA0
https://api.github.com/users/lewtun/orgs
https://api.github.com/users/lewtun/received_events
https://api.github.com/users/lewtun/repos
https://api.github.com/users/lewtun/starred{/owner}{/repo}
https://api.github.com/users/lewtun/subscriptions
User
https://api.github.com/users/lewtun
public
MEMBER
> Good!
> 
> I have a suggested refactoring... Tell me what you think! :)

your approach is much more elegant - i've included your suggestions 🙏 
2021-07-13T10:00:34Z
https://github.com/huggingface/datasets/pull/2619#issuecomment-878951598
https://api.github.com/repos/huggingface/datasets/issues/2619
MDEyOklzc3VlQ29tbWVudDg3ODk1MTU5OA==
https://api.github.com/repos/huggingface/datasets/issues/comments/878951598/reactions
2021-07-13T10:00:34Z
https://api.github.com/repos/huggingface/datasets/issues/comments/878951598
https://avatars.githubusercontent.com/u/26859204?v=4
https://api.github.com/users/lewtun/events{/privacy}
https://api.github.com/users/lewtun/followers
https://api.github.com/users/lewtun/following{/other_user}
https://api.github.com/users/lewtun/gists{/gist_id}
https://github.com/lewtun
lewtun
MDQ6VXNlcjI2ODU5MjA0
https://api.github.com/users/lewtun/orgs
https://api.github.com/users/lewtun/received_events
https://api.github.com/users/lewtun/repos
https://api.github.com/users/lewtun/starred{/owner}{/repo}
https://api.github.com/users/lewtun/subscriptions
User
https://api.github.com/users/lewtun
public
This PR starts building up the SUPERB benchmark by including the ASR task as described in the [SUPERB paper](https://arxiv.org/abs/2105.01051) and `s3prl` [instructions](https://github.com/s3prl/s3prl/tree/v0.2.0/downstream#asr-automatic-speech-recognition).

Usage:

```python
from datasets import load_dataset 

asr = load_dataset("superb", "asr")
# DatasetDict({
#     train: Dataset({
#         features: ['file', 'text', 'speaker_id', 'chapter_id', 'id'],
#         num_rows: 28539
#     })
#     validation: Dataset({
#         features: ['file', 'text', 'speaker_id', 'chapter_id', 'id'],
#         num_rows: 2703
#     })
#     test: Dataset({
#         features: ['file', 'text', 'speaker_id', 'chapter_id', 'id'],
#         num_rows: 2620
#     })
# })
```

I've used the GLUE benchmark as a guide for filling out the README.

To move fast during the evaluation PoC I propose to merge one task at a time, so we can continue building the training / evaluation framework in parallel.

Note: codewise this PR is ready for review - I'll add the missing YAML tags once #2620 is merged :)