Transform references for sacrebleu
CONTRIBUTOR
I think I agree @lhoestq so I pushed a change.
Thanks for your work on the library!
2020-08-20T09:02:09Z
https://github.com/huggingface/datasets/pull/520#issuecomment-677473416
https://api.github.com/repos/huggingface/datasets/issues/520
MDEyOklzc3VlQ29tbWVudDY3NzQ3MzQxNg==
https://api.github.com/repos/huggingface/datasets/issues/comments/677473416/reactions
2020-08-20T09:02:09Z
https://api.github.com/repos/huggingface/datasets/issues/comments/677473416
https://avatars.githubusercontent.com/u/2238344?v=4
https://api.github.com/users/jbragg/events{/privacy}
https://api.github.com/users/jbragg/followers
https://api.github.com/users/jbragg/following{/other_user}
https://api.github.com/users/jbragg/gists{/gist_id}
https://github.com/jbragg
jbragg
MDQ6VXNlcjIyMzgzNDQ=
https://api.github.com/users/jbragg/orgs
https://api.github.com/users/jbragg/received_events
https://api.github.com/users/jbragg/repos
https://api.github.com/users/jbragg/starred{/owner}{/repo}
https://api.github.com/users/jbragg/subscriptions
User
https://api.github.com/users/jbragg
public
Currently it is impossible to use sacrebleu when len(predictions) != the number of references per prediction (very uncommon), due to a strange format expected by sacrebleu. If one passes in the data to `nlp.metric.compute()` in sacrebleu format, `nlp` throws an error due to mismatching lengths between predictions and references. If one uses a more standard format where predictions and references are lists of the same length, sacrebleu throws an error.

This PR transforms reference data in a more standard format into the [unusual format](https://github.com/mjpost/sacreBLEU#using-sacrebleu-from-python) expected by sacrebleu.