Fixes FrugalScore
NONE
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_4305). All of your documentation changes will be reflected on that endpoint.
2022-05-10T12:55:43Z
https://github.com/huggingface/datasets/pull/4305#issuecomment-1122354328
https://api.github.com/repos/huggingface/datasets/issues/4305
IC_kwDODunzps5C5cSY
https://api.github.com/repos/huggingface/datasets/issues/comments/1122354328/reactions
2022-05-10T12:55:43Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1122354328
https://avatars.githubusercontent.com/u/99929124?v=4
https://api.github.com/users/HuggingFaceDocBuilderDev/events{/privacy}
https://api.github.com/users/HuggingFaceDocBuilderDev/followers
https://api.github.com/users/HuggingFaceDocBuilderDev/following{/other_user}
https://api.github.com/users/HuggingFaceDocBuilderDev/gists{/gist_id}
https://github.com/HuggingFaceDocBuilderDev
HuggingFaceDocBuilderDev
U_kgDOBfTMJA
https://api.github.com/users/HuggingFaceDocBuilderDev/orgs
https://api.github.com/users/HuggingFaceDocBuilderDev/received_events
https://api.github.com/users/HuggingFaceDocBuilderDev/repos
https://api.github.com/users/HuggingFaceDocBuilderDev/starred{/owner}{/repo}
https://api.github.com/users/HuggingFaceDocBuilderDev/subscriptions
User
https://api.github.com/users/HuggingFaceDocBuilderDev
public
MEMBER
> predictions and references are swapped. Basically Frugalscore is commutative, however some tiny differences can occur if we swap the references and the predictions. I decided to swap them just to obtain the exact results as reported in the paper.

What is the order of magnitude of the difference ? Do you know what causes this ?

> I switched to dynamic padding that was was used in the training, forcing the padding to max_length introduces errors for some reason that I ignore.

What error ?
2022-05-11T08:49:12Z
https://github.com/huggingface/datasets/pull/4305#issuecomment-1123380233
https://api.github.com/repos/huggingface/datasets/issues/4305
IC_kwDODunzps5C9WwJ
https://api.github.com/repos/huggingface/datasets/issues/comments/1123380233/reactions
2022-05-11T08:49:12Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1123380233
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
There are two minor modifications in this PR:
1) `predictions` and `references` are swapped. Basically Frugalscore is commutative, however some tiny differences can occur if we swap the references and the predictions. I decided to swap them just to obtain the exact results as reported in the paper.
2) I switched to dynamic padding that was was used in the training, forcing the padding to `max_length`  introduces errors for some reason that I ignore.

@lhoestq 