Add rouge-2 in rouge_types for metric calculation
MEMBER
Indeed there's currently a mismatch between the description and what it rouge actually returns.
Thanks for proposing this fix :) 

I think it's better to return rouge 1-2-L.
Was there a reason to only include rouge 1 and rouge L @thomwolf ? 
2020-10-02T08:45:22Z
https://github.com/huggingface/datasets/pull/700#issuecomment-702605650
https://api.github.com/repos/huggingface/datasets/issues/700
MDEyOklzc3VlQ29tbWVudDcwMjYwNTY1MA==
https://api.github.com/repos/huggingface/datasets/issues/comments/702605650/reactions
2020-10-02T08:45:22Z
https://api.github.com/repos/huggingface/datasets/issues/comments/702605650
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
rougeLsum is also missing, could you add it ?
2020-10-02T09:26:22Z
https://github.com/huggingface/datasets/pull/700#issuecomment-702624100
https://api.github.com/repos/huggingface/datasets/issues/700
MDEyOklzc3VlQ29tbWVudDcwMjYyNDEwMA==
https://api.github.com/repos/huggingface/datasets/issues/comments/702624100/reactions
2020-10-02T09:26:22Z
https://api.github.com/repos/huggingface/datasets/issues/comments/702624100
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
Adding `RougeLSum` would fix https://github.com/huggingface/datasets/issues/617
2020-10-02T09:27:00Z
https://github.com/huggingface/datasets/pull/700#issuecomment-702624396
https://api.github.com/repos/huggingface/datasets/issues/700
MDEyOklzc3VlQ29tbWVudDcwMjYyNDM5Ng==
https://api.github.com/repos/huggingface/datasets/issues/comments/702624396/reactions
2020-10-02T09:27:00Z
https://api.github.com/repos/huggingface/datasets/issues/comments/702624396
https://avatars.githubusercontent.com/u/7353373?v=4
https://api.github.com/users/thomwolf/events{/privacy}
https://api.github.com/users/thomwolf/followers
https://api.github.com/users/thomwolf/following{/other_user}
https://api.github.com/users/thomwolf/gists{/gist_id}
https://github.com/thomwolf
thomwolf
MDQ6VXNlcjczNTMzNzM=
https://api.github.com/users/thomwolf/orgs
https://api.github.com/users/thomwolf/received_events
https://api.github.com/users/thomwolf/repos
https://api.github.com/users/thomwolf/starred{/owner}{/repo}
https://api.github.com/users/thomwolf/subscriptions
User
https://api.github.com/users/thomwolf
public
MEMBER
I am opening a PR with both of them right now actually :)
2020-10-02T09:28:13Z
https://github.com/huggingface/datasets/pull/700#issuecomment-702624956
https://api.github.com/repos/huggingface/datasets/issues/700
MDEyOklzc3VlQ29tbWVudDcwMjYyNDk1Ng==
https://api.github.com/repos/huggingface/datasets/issues/comments/702624956/reactions
2020-10-02T09:28:13Z
https://api.github.com/repos/huggingface/datasets/issues/comments/702624956
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
Also the format of the output isn't exactly ideal, It's usually only the F-1 score that is cared about. 

Formatting the output to reflect how `ROUGE-1-5-5` (the perl version thats usually used and pyrouge is a wrapper over it), would be better.


2020-10-02T09:50:15Z
https://github.com/huggingface/datasets/pull/700#issuecomment-702634847
https://api.github.com/repos/huggingface/datasets/issues/700
MDEyOklzc3VlQ29tbWVudDcwMjYzNDg0Nw==
https://api.github.com/repos/huggingface/datasets/issues/comments/702634847/reactions
2020-10-02T09:58:50Z
https://api.github.com/repos/huggingface/datasets/issues/comments/702634847
https://avatars.githubusercontent.com/u/18056781?v=4
https://api.github.com/users/Shashi456/events{/privacy}
https://api.github.com/users/Shashi456/followers
https://api.github.com/users/Shashi456/following{/other_user}
https://api.github.com/users/Shashi456/gists{/gist_id}
https://github.com/Shashi456
Shashi456
MDQ6VXNlcjE4MDU2Nzgx
https://api.github.com/users/Shashi456/orgs
https://api.github.com/users/Shashi456/received_events
https://api.github.com/users/Shashi456/repos
https://api.github.com/users/Shashi456/starred{/owner}{/repo}
https://api.github.com/users/Shashi456/subscriptions
User
https://api.github.com/users/Shashi456
public
NONE
I'll close this since you seem to have already added it in another PR. Sorry for the delay in responding to you @lhoestq.
2020-10-02T09:59:05Z
https://github.com/huggingface/datasets/pull/700#issuecomment-702639332
https://api.github.com/repos/huggingface/datasets/issues/700
MDEyOklzc3VlQ29tbWVudDcwMjYzOTMzMg==
https://api.github.com/repos/huggingface/datasets/issues/comments/702639332/reactions
2020-10-02T09:59:05Z
https://api.github.com/repos/huggingface/datasets/issues/comments/702639332
https://avatars.githubusercontent.com/u/18056781?v=4
https://api.github.com/users/Shashi456/events{/privacy}
https://api.github.com/users/Shashi456/followers
https://api.github.com/users/Shashi456/following{/other_user}
https://api.github.com/users/Shashi456/gists{/gist_id}
https://github.com/Shashi456
Shashi456
MDQ6VXNlcjE4MDU2Nzgx
https://api.github.com/users/Shashi456/orgs
https://api.github.com/users/Shashi456/received_events
https://api.github.com/users/Shashi456/repos
https://api.github.com/users/Shashi456/starred{/owner}{/repo}
https://api.github.com/users/Shashi456/subscriptions
User
https://api.github.com/users/Shashi456
public
MEMBER
What do you mean by "Formatting the output to reflect how ROUGE-1-5-5" @Shashi456 ?
2020-10-02T10:03:25Z
https://github.com/huggingface/datasets/pull/700#issuecomment-702641575
https://api.github.com/repos/huggingface/datasets/issues/700
MDEyOklzc3VlQ29tbWVudDcwMjY0MTU3NQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/702641575/reactions
2020-10-02T10:03:25Z
https://api.github.com/repos/huggingface/datasets/issues/comments/702641575
https://avatars.githubusercontent.com/u/7353373?v=4
https://api.github.com/users/thomwolf/events{/privacy}
https://api.github.com/users/thomwolf/followers
https://api.github.com/users/thomwolf/following{/other_user}
https://api.github.com/users/thomwolf/gists{/gist_id}
https://github.com/thomwolf
thomwolf
MDQ6VXNlcjczNTMzNzM=
https://api.github.com/users/thomwolf/orgs
https://api.github.com/users/thomwolf/received_events
https://api.github.com/users/thomwolf/repos
https://api.github.com/users/thomwolf/starred{/owner}{/repo}
https://api.github.com/users/thomwolf/subscriptions
User
https://api.github.com/users/thomwolf
public
MEMBER
I like the idea of returning all the scores for two reason:
- Rouge's aggregator does sampling and therefore it returns "low" "mid" and "high" scores
- It is interesting to have the precision and recall to see how the F1 score was computed
But I understand your point that returning only the F1 score makes sense since it's the one that's always used 
2020-10-02T10:05:07Z
https://github.com/huggingface/datasets/pull/700#issuecomment-702642466
https://api.github.com/repos/huggingface/datasets/issues/700
MDEyOklzc3VlQ29tbWVudDcwMjY0MjQ2Ng==
https://api.github.com/repos/huggingface/datasets/issues/comments/702642466/reactions
2020-10-02T10:05:07Z
https://api.github.com/repos/huggingface/datasets/issues/comments/702642466
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
@thomwolf the scores now returned look like this:
```
{'rouge1': AggregateScore(low=Score(precision=0.16620308156871524, recall=0.18219819615984395, fmeasure=0.16226017699359463), mid=Score(precision=0.17274338501705871, recall=0.1890957812369246, fmeasure=0.16823877588620403), high=Score(precision=0.17934569582981455, recall=0.1965626706042028, fmeasure=0.17491509794856058)), 
'rouge2': AggregateScore(low=Score(precision=0.12478835737689957, recall=0.1362113231755514, fmeasure=0.12055941950062395), mid=Score(precision=0.1303967602691664, recall=0.1423747229852964, fmeasure=0.1258363976151122), high=Score(precision=0.13654527560789362, recall=0.1488071465116122, fmeasure=0.13184989406704056)), 
'rougeL': AggregateScore(low=Score(precision=0.16568068818352072, recall=0.1811919016674486, fmeasure=0.1614784523482225), mid=Score(precision=0.17156684723552357, recall=0.1879777628247058, fmeasure=0.16720699286250762), high=Score(precision=0.17788847350584547, recall=0.1948899838530898, fmeasure=0.17316501523379826))}
```

While when computed through the perl rouge script, it looks like:
```
ROUGE-1 Average_R: 0.34775 (95%-conf.int. 0.34546 - 0.35025)
ROUGE-1 Average_P: 0.19381 (95%-conf.int. 0.19246 - 0.19538)
ROUGE-1 Average_F: 0.24070 (95%-conf.int. 0.23925 - 0.24230)
---------------------------------------------
ROUGE-2 Average_R: 0.07160 (95%-conf.int. 0.07010 - 0.07298)
ROUGE-2 Average_F: 0.04845 (95%-conf.int. 0.04741 - 0.04942)
---------------------------------------------
ROUGE-L Average_R: 0.26404 (95%-conf.int. 0.26215 - 0.26598)
ROUGE-L Average_P: 0.14696 (95%-conf.int. 0.14576 - 0.14815)
ROUGE-L Average_F: 0.18245 (95%-conf.int. 0.18120 - 0.18367)
```
while the wrapper returns the much more readable:
```
[2020-07-30 18:13:38,556 INFO] Rouges at step 13000                                       
>> ROUGE-F(1/2/3/l): 43.43/20.42/39.78              
ROUGE-R(1/2/3/l): 53.91/25.34/49.32
```

The formatting allows for easy reading, and although "low", "mid", "high" make sense, this is more concise and effective. 

One way of changing this might be to return a dictionary that returns values like `rouge_1_precision`, `rouge_1_F1`, `rouge_1_recall`, and maybe also having the ability to get the values you are interested in and keeping `recall` and `F1` as default.
2020-10-02T10:15:00Z
https://github.com/huggingface/datasets/pull/700#issuecomment-702647308
https://api.github.com/repos/huggingface/datasets/issues/700
MDEyOklzc3VlQ29tbWVudDcwMjY0NzMwOA==
https://api.github.com/repos/huggingface/datasets/issues/comments/702647308/reactions
2020-10-02T10:20:24Z
https://api.github.com/repos/huggingface/datasets/issues/comments/702647308
https://avatars.githubusercontent.com/u/18056781?v=4
https://api.github.com/users/Shashi456/events{/privacy}
https://api.github.com/users/Shashi456/followers
https://api.github.com/users/Shashi456/following{/other_user}
https://api.github.com/users/Shashi456/gists{/gist_id}
https://github.com/Shashi456
Shashi456
MDQ6VXNlcjE4MDU2Nzgx
https://api.github.com/users/Shashi456/orgs
https://api.github.com/users/Shashi456/received_events
https://api.github.com/users/Shashi456/repos
https://api.github.com/users/Shashi456/starred{/owner}{/repo}
https://api.github.com/users/Shashi456/subscriptions
User
https://api.github.com/users/Shashi456
public
NONE
cc: @lhoestq 
2020-10-02T10:18:09Z
https://github.com/huggingface/datasets/pull/700#issuecomment-702648821
https://api.github.com/repos/huggingface/datasets/issues/700
MDEyOklzc3VlQ29tbWVudDcwMjY0ODgyMQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/702648821/reactions
2020-10-02T10:18:09Z
https://api.github.com/repos/huggingface/datasets/issues/comments/702648821
https://avatars.githubusercontent.com/u/18056781?v=4
https://api.github.com/users/Shashi456/events{/privacy}
https://api.github.com/users/Shashi456/followers
https://api.github.com/users/Shashi456/following{/other_user}
https://api.github.com/users/Shashi456/gists{/gist_id}
https://github.com/Shashi456
Shashi456
MDQ6VXNlcjE4MDU2Nzgx
https://api.github.com/users/Shashi456/orgs
https://api.github.com/users/Shashi456/received_events
https://api.github.com/users/Shashi456/repos
https://api.github.com/users/Shashi456/starred{/owner}{/repo}
https://api.github.com/users/Shashi456/subscriptions
User
https://api.github.com/users/Shashi456
public
MEMBER
Ok I see.
I think it's also important to follow one of the existing output format (there are already too many different formats, let's try not to add another different one)
I'd still stick with the current format and not transform the output of the python implementation of rouge since it's already widely used.
What do you think ?
2020-10-02T10:33:20Z
https://github.com/huggingface/datasets/pull/700#issuecomment-702656091
https://api.github.com/repos/huggingface/datasets/issues/700
MDEyOklzc3VlQ29tbWVudDcwMjY1NjA5MQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/702656091/reactions
2020-10-02T10:33:20Z
https://api.github.com/repos/huggingface/datasets/issues/comments/702656091
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
Maybe we could convert the dataclasses in dictionnaries, would that help @Shashi456 ?
2020-10-02T10:36:15Z
https://github.com/huggingface/datasets/pull/700#issuecomment-702657350
https://api.github.com/repos/huggingface/datasets/issues/700
MDEyOklzc3VlQ29tbWVudDcwMjY1NzM1MA==
https://api.github.com/repos/huggingface/datasets/issues/comments/702657350/reactions
2020-10-02T10:36:15Z
https://api.github.com/repos/huggingface/datasets/issues/comments/702657350
https://avatars.githubusercontent.com/u/7353373?v=4
https://api.github.com/users/thomwolf/events{/privacy}
https://api.github.com/users/thomwolf/followers
https://api.github.com/users/thomwolf/following{/other_user}
https://api.github.com/users/thomwolf/gists{/gist_id}
https://github.com/thomwolf
thomwolf
MDQ6VXNlcjczNTMzNzM=
https://api.github.com/users/thomwolf/orgs
https://api.github.com/users/thomwolf/received_events
https://api.github.com/users/thomwolf/repos
https://api.github.com/users/thomwolf/starred{/owner}{/repo}
https://api.github.com/users/thomwolf/subscriptions
User
https://api.github.com/users/thomwolf
public
NONE
@thomwolf yeah I think that would help. I initially didn't understand the high low mid categories. Dictionaries could help in this case I guess, and if we allow the user to choose what they want i.e F1 and precision or recall.
2020-10-02T11:07:56Z
https://github.com/huggingface/datasets/pull/700#issuecomment-702671435
https://api.github.com/repos/huggingface/datasets/issues/700
MDEyOklzc3VlQ29tbWVudDcwMjY3MTQzNQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/702671435/reactions
2020-10-02T11:08:49Z
https://api.github.com/repos/huggingface/datasets/issues/comments/702671435
https://avatars.githubusercontent.com/u/18056781?v=4
https://api.github.com/users/Shashi456/events{/privacy}
https://api.github.com/users/Shashi456/followers
https://api.github.com/users/Shashi456/following{/other_user}
https://api.github.com/users/Shashi456/gists{/gist_id}
https://github.com/Shashi456
Shashi456
MDQ6VXNlcjE4MDU2Nzgx
https://api.github.com/users/Shashi456/orgs
https://api.github.com/users/Shashi456/received_events
https://api.github.com/users/Shashi456/repos
https://api.github.com/users/Shashi456/starred{/owner}{/repo}
https://api.github.com/users/Shashi456/subscriptions
User
https://api.github.com/users/Shashi456
public
The description of the ROUGE metric says, 
```
_KWARGS_DESCRIPTION = """
Calculates average rouge scores for a list of hypotheses and references
Args:
    predictions: list of predictions to score. Each predictions
        should be a string with tokens separated by spaces.
    references: list of reference for each prediction. Each
        reference should be a string with tokens separated by spaces.
Returns:
    rouge1: rouge_1 f1,
    rouge2: rouge_2 f1,
    rougeL: rouge_l f1,
    rougeLsum: rouge_l precision
"""
```

but the `rouge_types` argument defaults to  `rouge_types = ["rouge1", "rougeL"]`, this PR updates and add `rouge2` to the list so as to reflect the description card.