Add bleurt
NONE
Sorry one nit: Could we use named arguments for the call to BLEURT?

i.e. 
        scores = self.scorer.score(references=references, candidates=predictions)

(i.e. so it is less bug prone)

2020-07-30T14:44:02Z
https://github.com/huggingface/datasets/pull/455#issuecomment-666414514
https://api.github.com/repos/huggingface/datasets/issues/455
MDEyOklzc3VlQ29tbWVudDY2NjQxNDUxNA==
https://api.github.com/repos/huggingface/datasets/issues/comments/666414514/reactions
2020-07-30T14:44:02Z
https://api.github.com/repos/huggingface/datasets/issues/comments/666414514
https://avatars.githubusercontent.com/u/31585795?v=4
https://api.github.com/users/ankparikh/events{/privacy}
https://api.github.com/users/ankparikh/followers
https://api.github.com/users/ankparikh/following{/other_user}
https://api.github.com/users/ankparikh/gists{/gist_id}
https://github.com/ankparikh
ankparikh
MDQ6VXNlcjMxNTg1Nzk1
https://api.github.com/users/ankparikh/orgs
https://api.github.com/users/ankparikh/received_events
https://api.github.com/users/ankparikh/repos
https://api.github.com/users/ankparikh/starred{/owner}{/repo}
https://api.github.com/users/ankparikh/subscriptions
User
https://api.github.com/users/ankparikh
public
NONE
Following up on Ankur's comment---we are going to drop support for
positional (not named) arguments in the future releases because it seems to
cause bugs and confusion. I hope it doesn't create too much of a mess.

Le jeu. 30 juil. 2020 à 10:44, ankparikh <notifications@github.com> a
écrit :

> Sorry one nit: Could we use named arguments for the call to BLEURT?
>
> i.e.
> scores = self.scorer.score(references=references, candidates=predictions)
>
> (i.e. so it is less bug prone)
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/nlp/pull/455#issuecomment-666414514>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ABTMRNGAN2PMECS5K4DIHJDR6GBMLANCNFSM4PL323FA>
> .
>

2020-07-30T15:25:50Z
https://github.com/huggingface/datasets/pull/455#issuecomment-666454460
https://api.github.com/repos/huggingface/datasets/issues/455
MDEyOklzc3VlQ29tbWVudDY2NjQ1NDQ2MA==
https://api.github.com/repos/huggingface/datasets/issues/comments/666454460/reactions
2020-07-30T15:25:50Z
https://api.github.com/repos/huggingface/datasets/issues/comments/666454460
https://avatars.githubusercontent.com/u/6736052?v=4
https://api.github.com/users/tsellam/events{/privacy}
https://api.github.com/users/tsellam/followers
https://api.github.com/users/tsellam/following{/other_user}
https://api.github.com/users/tsellam/gists{/gist_id}
https://github.com/tsellam
tsellam
MDQ6VXNlcjY3MzYwNTI=
https://api.github.com/users/tsellam/orgs
https://api.github.com/users/tsellam/received_events
https://api.github.com/users/tsellam/repos
https://api.github.com/users/tsellam/starred{/owner}{/repo}
https://api.github.com/users/tsellam/subscriptions
User
https://api.github.com/users/tsellam
public
MEMBER
> Following up on Ankur's comment---we are going to drop support for positional (not named) arguments in the future releases because it seems to cause bugs and confusion. I hope it doesn't create too much of a mess. Le jeu. 30 juil. 2020 à 10:44, ankparikh <notifications@github.com> a écrit :
> […](#)
> Sorry one nit: Could we use named arguments for the call to BLEURT? i.e. scores = self.scorer.score(references=references, candidates=predictions) (i.e. so it is less bug prone) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <[#455 (comment)](https://github.com/huggingface/nlp/pull/455#issuecomment-666414514)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABTMRNGAN2PMECS5K4DIHJDR6GBMLANCNFSM4PL323FA> .

Changed @ankparikh @tsellam, thanks for taking a look!
2020-07-30T15:28:59Z
https://github.com/huggingface/datasets/pull/455#issuecomment-666456332
https://api.github.com/repos/huggingface/datasets/issues/455
MDEyOklzc3VlQ29tbWVudDY2NjQ1NjMzMg==
https://api.github.com/repos/huggingface/datasets/issues/comments/666456332/reactions
2020-07-30T15:28:59Z
https://api.github.com/repos/huggingface/datasets/issues/comments/666456332
https://avatars.githubusercontent.com/u/10469459?v=4
https://api.github.com/users/yjernite/events{/privacy}
https://api.github.com/users/yjernite/followers
https://api.github.com/users/yjernite/following{/other_user}
https://api.github.com/users/yjernite/gists{/gist_id}
https://github.com/yjernite
yjernite
MDQ6VXNlcjEwNDY5NDU5
https://api.github.com/users/yjernite/orgs
https://api.github.com/users/yjernite/received_events
https://api.github.com/users/yjernite/repos
https://api.github.com/users/yjernite/starred{/owner}{/repo}
https://api.github.com/users/yjernite/subscriptions
User
https://api.github.com/users/yjernite
public
MEMBER
We should avoid positional arguments in metrics on our side as well. It's a dangerous source of errors indeed.
2020-07-30T15:36:43Z
https://github.com/huggingface/datasets/pull/455#issuecomment-666465658
https://api.github.com/repos/huggingface/datasets/issues/455
MDEyOklzc3VlQ29tbWVudDY2NjQ2NTY1OA==
https://api.github.com/repos/huggingface/datasets/issues/comments/666465658/reactions
2020-07-30T15:36:43Z
https://api.github.com/repos/huggingface/datasets/issues/comments/666465658
https://avatars.githubusercontent.com/u/7353373?v=4
https://api.github.com/users/thomwolf/events{/privacy}
https://api.github.com/users/thomwolf/followers
https://api.github.com/users/thomwolf/following{/other_user}
https://api.github.com/users/thomwolf/gists{/gist_id}
https://github.com/thomwolf
thomwolf
MDQ6VXNlcjczNTMzNzM=
https://api.github.com/users/thomwolf/orgs
https://api.github.com/users/thomwolf/received_events
https://api.github.com/users/thomwolf/repos
https://api.github.com/users/thomwolf/starred{/owner}{/repo}
https://api.github.com/users/thomwolf/subscriptions
User
https://api.github.com/users/thomwolf
public
This PR adds the BLEURT metric to the library.

The BLEURT `Metric` downloads a TF checkpoint corresponding to its `config_name` at creation (in the `_info` function). Default is set to `bleurt-base-128`.

Note that the default in the original package is `bleurt-tiny-128`, but they throw a warning and recommend using `bleurt-base-128` instead. I think it's safer to have our users have a functioning metric when they call the default behavior, we'll address discrepancies in the issues/discussions if it comes up.

In addition to the BLEURT file, `load.py` was changed so we can ask users to pip install the required packages from git when they have a `setup.py` but are not on PyPL

cc @ankparikh @tsellam