Compute WER metric iteratively
MEMBER
I discussed with Patrick and I think we could have a nice addition: have a parameter `concatenate_texts` that, if `True`, uses the old implementation.

By default `concatenate_texts` would be `False`, so that sentences are evaluated independently, and to save resources (the WER computation has a quadratic complexity).

Some users might still want to use the old implementation.
2021-03-31T08:46:08Z
https://github.com/huggingface/datasets/pull/2111#issuecomment-810892818
https://api.github.com/repos/huggingface/datasets/issues/2111
MDEyOklzc3VlQ29tbWVudDgxMDg5MjgxOA==
https://api.github.com/repos/huggingface/datasets/issues/comments/810892818/reactions
2021-03-31T08:46:08Z
https://api.github.com/repos/huggingface/datasets/issues/comments/810892818
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
@lhoestq @patrickvonplaten are you sure of the parameter name `concatenate_texts`? I was thinking about something like `iter`...
2021-04-01T08:47:40Z
https://github.com/huggingface/datasets/pull/2111#issuecomment-811755937
https://api.github.com/repos/huggingface/datasets/issues/2111
MDEyOklzc3VlQ29tbWVudDgxMTc1NTkzNw==
https://api.github.com/repos/huggingface/datasets/issues/comments/811755937/reactions
2021-04-01T08:47:40Z
https://api.github.com/repos/huggingface/datasets/issues/comments/811755937
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
MEMBER
Not sure about the name, if you can improve it feel free to do so ^^'
The old implementation computes the WER on the concatenation of all the input texts, while the new one makes WER measures computation independent for each reference/prediction pair.
That's why I thought of `concatenate_texts`
2021-04-01T08:49:58Z
https://github.com/huggingface/datasets/pull/2111#issuecomment-811757524
https://api.github.com/repos/huggingface/datasets/issues/2111
MDEyOklzc3VlQ29tbWVudDgxMTc1NzUyNA==
https://api.github.com/repos/huggingface/datasets/issues/comments/811757524/reactions
2021-04-01T08:52:57Z
https://api.github.com/repos/huggingface/datasets/issues/comments/811757524
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
@lhoestq yes, but the end user does not necessarily know the details of the implementation of the WER computation.

From the end user perspective I think it might make more sense: how do you want to compute the metric?
- all in once, more RAM memory needed?
- iteratively, less RAM requirements?

Because of that I was thinking of something like `iter` or `iterative`...
2021-04-01T10:02:08Z
https://github.com/huggingface/datasets/pull/2111#issuecomment-811800373
https://api.github.com/repos/huggingface/datasets/issues/2111
MDEyOklzc3VlQ29tbWVudDgxMTgwMDM3Mw==
https://api.github.com/repos/huggingface/datasets/issues/comments/811800373/reactions
2021-04-01T10:02:08Z
https://api.github.com/repos/huggingface/datasets/issues/comments/811800373
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
MEMBER
Personally like `concatenate_texts` better since I feel like `iter` or `iterate` are quite vague
2021-04-06T07:12:44Z
https://github.com/huggingface/datasets/pull/2111#issuecomment-813884681
https://api.github.com/repos/huggingface/datasets/issues/2111
MDEyOklzc3VlQ29tbWVudDgxMzg4NDY4MQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/813884681/reactions
2021-04-06T07:12:44Z
https://api.github.com/repos/huggingface/datasets/issues/comments/813884681
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
Therefore, you can merge... ;)
2021-04-06T07:19:29Z
https://github.com/huggingface/datasets/pull/2111#issuecomment-813888760
https://api.github.com/repos/huggingface/datasets/issues/2111
MDEyOklzc3VlQ29tbWVudDgxMzg4ODc2MA==
https://api.github.com/repos/huggingface/datasets/issues/comments/813888760/reactions
2021-04-06T07:19:29Z
https://api.github.com/repos/huggingface/datasets/issues/comments/813888760
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
MEMBER
Ok ! merging :)
2021-04-06T07:20:38Z
https://github.com/huggingface/datasets/pull/2111#issuecomment-813889457
https://api.github.com/repos/huggingface/datasets/issues/2111
MDEyOklzc3VlQ29tbWVudDgxMzg4OTQ1Nw==
https://api.github.com/repos/huggingface/datasets/issues/comments/813889457/reactions
2021-04-06T07:20:38Z
https://api.github.com/repos/huggingface/datasets/issues/comments/813889457
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
Compute WER metric iteratively to avoid MemoryError.

Fix #2078.