Squad Metric Description & Feature Mismatch
MEMBER
Thanks for reporting !
There indeed a mismatch between the features and the kwargs description

I believe `answer_start` was added to match the squad dataset format for consistency, even though it is not used in the metric computation. I think I'd rather keep it this way, so that you can just give `references=squad["answers"]` to `.compute()`.
Maybe we can just fix the description then.
2020-09-22T10:17:21Z
https://github.com/huggingface/datasets/issues/657#issuecomment-696632219
https://api.github.com/repos/huggingface/datasets/issues/657
MDEyOklzc3VlQ29tbWVudDY5NjYzMjIxOQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/696632219/reactions
2020-09-22T10:17:21Z
https://api.github.com/repos/huggingface/datasets/issues/comments/696632219
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
But then providing the `answer_start` becomes mandatory since the format of the features is checked against the one provided in the squad [file](https://github.com/huggingface/datasets/pull/658/files).
2020-10-13T02:16:56Z
https://github.com/huggingface/datasets/issues/657#issuecomment-707442085
https://api.github.com/repos/huggingface/datasets/issues/657
MDEyOklzc3VlQ29tbWVudDcwNzQ0MjA4NQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/707442085/reactions
2020-10-13T02:16:56Z
https://api.github.com/repos/huggingface/datasets/issues/comments/707442085
https://avatars.githubusercontent.com/u/8372098?v=4
https://api.github.com/users/tshrjn/events{/privacy}
https://api.github.com/users/tshrjn/followers
https://api.github.com/users/tshrjn/following{/other_user}
https://api.github.com/users/tshrjn/gists{/gist_id}
https://github.com/tshrjn
tshrjn
MDQ6VXNlcjgzNzIwOTg=
https://api.github.com/users/tshrjn/orgs
https://api.github.com/users/tshrjn/received_events
https://api.github.com/users/tshrjn/repos
https://api.github.com/users/tshrjn/starred{/owner}{/repo}
https://api.github.com/users/tshrjn/subscriptions
User
https://api.github.com/users/tshrjn
public
The [description](https://github.com/huggingface/datasets/blob/master/metrics/squad/squad.py#L39) doesn't mention `answer_start` in squad. However the `datasets.features` require [it](https://github.com/huggingface/datasets/blob/master/metrics/squad/squad.py#L68). It's also not used in the evaluation.