Multilabel metrics not supported
MEMBER
Hi @GuillemGSubies, thanks for reporting.

I have made a PR to fix this issue and allow metrics to be computed also for multilabel classification problems.
2021-07-05T09:31:58Z
https://github.com/huggingface/datasets/issues/2554#issuecomment-873962349
https://api.github.com/repos/huggingface/datasets/issues/2554
MDEyOklzc3VlQ29tbWVudDg3Mzk2MjM0OQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/873962349/reactions
2021-07-05T09:31:58Z
https://api.github.com/repos/huggingface/datasets/issues/comments/873962349
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
NONE
Looks nice, thank you very much! 🚀 
2021-07-05T10:55:10Z
https://github.com/huggingface/datasets/issues/2554#issuecomment-874017116
https://api.github.com/repos/huggingface/datasets/issues/2554
MDEyOklzc3VlQ29tbWVudDg3NDAxNzExNg==
https://api.github.com/repos/huggingface/datasets/issues/comments/874017116/reactions
2021-07-05T10:55:10Z
https://api.github.com/repos/huggingface/datasets/issues/comments/874017116
https://avatars.githubusercontent.com/u/37592763?v=4
https://api.github.com/users/GuillemGSubies/events{/privacy}
https://api.github.com/users/GuillemGSubies/followers
https://api.github.com/users/GuillemGSubies/following{/other_user}
https://api.github.com/users/GuillemGSubies/gists{/gist_id}
https://github.com/GuillemGSubies
GuillemGSubies
MDQ6VXNlcjM3NTkyNzYz
https://api.github.com/users/GuillemGSubies/orgs
https://api.github.com/users/GuillemGSubies/received_events
https://api.github.com/users/GuillemGSubies/repos
https://api.github.com/users/GuillemGSubies/starred{/owner}{/repo}
https://api.github.com/users/GuillemGSubies/subscriptions
User
https://api.github.com/users/GuillemGSubies
public
NONE
Sorry for reopening but I just noticed that the `_compute` method for the F1 metric is still not good enough for multilabel problems:

https://github.com/huggingface/datasets/blob/92a3ee549705aa0a107c9fa5caf463b3b3da2616/metrics/f1/f1.py#L115

Somehow we should be able to change the parameter `average` at least
2021-10-13T12:16:07Z
https://github.com/huggingface/datasets/issues/2554#issuecomment-942242369
https://api.github.com/repos/huggingface/datasets/issues/2554
IC_kwDODunzps44KXpB
https://api.github.com/repos/huggingface/datasets/issues/comments/942242369/reactions
2021-10-13T12:16:07Z
https://api.github.com/repos/huggingface/datasets/issues/comments/942242369
https://avatars.githubusercontent.com/u/37592763?v=4
https://api.github.com/users/GuillemGSubies/events{/privacy}
https://api.github.com/users/GuillemGSubies/followers
https://api.github.com/users/GuillemGSubies/following{/other_user}
https://api.github.com/users/GuillemGSubies/gists{/gist_id}
https://github.com/GuillemGSubies
GuillemGSubies
MDQ6VXNlcjM3NTkyNzYz
https://api.github.com/users/GuillemGSubies/orgs
https://api.github.com/users/GuillemGSubies/received_events
https://api.github.com/users/GuillemGSubies/repos
https://api.github.com/users/GuillemGSubies/starred{/owner}{/repo}
https://api.github.com/users/GuillemGSubies/subscriptions
User
https://api.github.com/users/GuillemGSubies
public
MEMBER
@GuillemGSubies, the parameter `average` passed to `_compute` is then passed to `f1_score`. This is right.
2021-10-13T12:29:13Z
https://github.com/huggingface/datasets/issues/2554#issuecomment-942253538
https://api.github.com/repos/huggingface/datasets/issues/2554
IC_kwDODunzps44KaXi
https://api.github.com/repos/huggingface/datasets/issues/comments/942253538/reactions
2021-10-13T12:29:13Z
https://api.github.com/repos/huggingface/datasets/issues/comments/942253538
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
When I try to use a metric like F1 macro I get the following error:

```
TypeError: int() argument must be a string, a bytes-like object or a number, not 'list'
```
There is an explicit casting here:

https://github.com/huggingface/datasets/blob/fc79f61cbbcfa0e8c68b28c0a8257f17e768a075/src/datasets/features.py#L274

And looks like this is because here

https://github.com/huggingface/datasets/blob/fc79f61cbbcfa0e8c68b28c0a8257f17e768a075/metrics/f1/f1.py#L88

the features can only be integers, so we cannot use that F1 for multilabel. Instead, if I create the following F1 (ints replaced with sequence of ints), it will work:

```python
class F1(datasets.Metric):
    def _info(self):
        return datasets.MetricInfo(
            description=_DESCRIPTION,
            citation=_CITATION,
            inputs_description=_KWARGS_DESCRIPTION,
            features=datasets.Features(
                {
                    "predictions": datasets.Sequence(datasets.Value("int32")),
                    "references": datasets.Sequence(datasets.Value("int32")),
                }
            ),
            reference_urls=["https://scikit-learn.org/stable/modules/generated/sklearn.metrics.f1_score.html"],
        )

    def _compute(self, predictions, references, labels=None, pos_label=1, average="binary", sample_weight=None):
        return {
            "f1": f1_score(
                references,
                predictions,
                labels=labels,
                pos_label=pos_label,
                average=average,
                sample_weight=sample_weight,
            ),
        }
```
