Allow loading community metrics from the hub, just like datasets
MEMBER
Hi ! Thanks for your message :) This is a great idea indeed. We haven't started working on this yet though. For now I guess you can host your metric on the Hub (either with your model or your dataset) and use `hf_hub_download` to download it (docs [here](https://github.com/huggingface/huggingface_hub/blob/main/docs/hub/how-to-downstream.md#cached_download))
2022-01-06T15:32:34Z
https://github.com/huggingface/datasets/issues/3543#issuecomment-1006683445
https://api.github.com/repos/huggingface/datasets/issues/3543
IC_kwDODunzps48AMU1
https://api.github.com/repos/huggingface/datasets/issues/comments/1006683445/reactions
2022-01-06T15:32:34Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1006683445
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
This is a great solution in the meantime, thanks!
2022-01-06T16:56:46Z
https://github.com/huggingface/datasets/issues/3543#issuecomment-1006747372
https://api.github.com/repos/huggingface/datasets/issues/3543
IC_kwDODunzps48Ab7s
https://api.github.com/repos/huggingface/datasets/issues/comments/1006747372/reactions
2022-01-06T16:56:46Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1006747372
https://avatars.githubusercontent.com/u/13485709?v=4
https://api.github.com/users/eladsegal/events{/privacy}
https://api.github.com/users/eladsegal/followers
https://api.github.com/users/eladsegal/following{/other_user}
https://api.github.com/users/eladsegal/gists{/gist_id}
https://github.com/eladsegal
eladsegal
MDQ6VXNlcjEzNDg1NzA5
https://api.github.com/users/eladsegal/orgs
https://api.github.com/users/eladsegal/received_events
https://api.github.com/users/eladsegal/repos
https://api.github.com/users/eladsegal/starred{/owner}{/repo}
https://api.github.com/users/eladsegal/subscriptions
User
https://api.github.com/users/eladsegal
public
CONTRIBUTOR
Here's the code I used, in case it can be of help to someone else:
```python
import os, shutil
from huggingface_hub import hf_hub_download
def download_metric(repo_id, file_path):
    # repo_id: for models "username/model_name", for datasets "datasets/username/model_name"
    local_metric_path = hf_hub_download(repo_id=repo_id, filename=file_path)
    updated_local_metric_path = (os.path.dirname(local_metric_path) + os.path.basename(local_metric_path).replace(".", "_") + ".py")
    shutil.copy(local_metric_path, updated_local_metric_path)
    return updated_local_metric_path

metric = load_metric(download_metric(REPO_ID, FILE_PATH))
```
2022-01-09T20:25:26Z
https://github.com/huggingface/datasets/issues/3543#issuecomment-1008367997
https://api.github.com/repos/huggingface/datasets/issues/3543
IC_kwDODunzps48Gnl9
https://api.github.com/repos/huggingface/datasets/issues/comments/1008367997/reactions
2022-01-09T20:28:13Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1008367997
https://avatars.githubusercontent.com/u/13485709?v=4
https://api.github.com/users/eladsegal/events{/privacy}
https://api.github.com/users/eladsegal/followers
https://api.github.com/users/eladsegal/following{/other_user}
https://api.github.com/users/eladsegal/gists{/gist_id}
https://github.com/eladsegal
eladsegal
MDQ6VXNlcjEzNDg1NzA5
https://api.github.com/users/eladsegal/orgs
https://api.github.com/users/eladsegal/received_events
https://api.github.com/users/eladsegal/repos
https://api.github.com/users/eladsegal/starred{/owner}{/repo}
https://api.github.com/users/eladsegal/subscriptions
User
https://api.github.com/users/eladsegal
public
CONTRIBUTOR
Solved with https://github.com/huggingface/evaluate 🤗 
2022-05-31T20:53:37Z
https://github.com/huggingface/datasets/issues/3543#issuecomment-1142630261
https://api.github.com/repos/huggingface/datasets/issues/3543
IC_kwDODunzps5EGyd1
https://api.github.com/repos/huggingface/datasets/issues/comments/1142630261/reactions
2022-05-31T20:53:37Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1142630261
https://avatars.githubusercontent.com/u/13485709?v=4
https://api.github.com/users/eladsegal/events{/privacy}
https://api.github.com/users/eladsegal/followers
https://api.github.com/users/eladsegal/following{/other_user}
https://api.github.com/users/eladsegal/gists{/gist_id}
https://github.com/eladsegal
eladsegal
MDQ6VXNlcjEzNDg1NzA5
https://api.github.com/users/eladsegal/orgs
https://api.github.com/users/eladsegal/received_events
https://api.github.com/users/eladsegal/repos
https://api.github.com/users/eladsegal/starred{/owner}{/repo}
https://api.github.com/users/eladsegal/subscriptions
User
https://api.github.com/users/eladsegal
public
MEMBER
Yay!! cc @lvwerra @sashavor @douwekiela 

Please share your feedback @eladsegal =)
2022-05-31T20:59:13Z
https://github.com/huggingface/datasets/issues/3543#issuecomment-1142634439
https://api.github.com/repos/huggingface/datasets/issues/3543
IC_kwDODunzps5EGzfH
https://api.github.com/repos/huggingface/datasets/issues/comments/1142634439/reactions
2022-05-31T20:59:13Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1142634439
https://avatars.githubusercontent.com/u/326577?v=4
https://api.github.com/users/julien-c/events{/privacy}
https://api.github.com/users/julien-c/followers
https://api.github.com/users/julien-c/following{/other_user}
https://api.github.com/users/julien-c/gists{/gist_id}
https://github.com/julien-c
julien-c
MDQ6VXNlcjMyNjU3Nw==
https://api.github.com/users/julien-c/orgs
https://api.github.com/users/julien-c/received_events
https://api.github.com/users/julien-c/repos
https://api.github.com/users/julien-c/starred{/owner}{/repo}
https://api.github.com/users/julien-c/subscriptions
User
https://api.github.com/users/julien-c
public
**Is your feature request related to a problem? Please describe.**
Currently, I can load a metric implemented by me by providing the local path to the file in `load_metric`.
However, there is no option to do it with the metric uploaded to the hub.
This means that if I want to allow other users to use it, they must download it first which makes the usage less smooth.

**Describe the solution you'd like**
Load metrics from the hub just like datasets are loaded.
In order to not break stuff, the convention can be to put the metric file in a "metrics" folder in the hub.

