Adding HendrycksTest dataset
CONTRIBUTOR
@lhoestq Thank you for the review. I've made the suggested changes. There still might be some problems with dummy data though due to some csv loading issues (which I haven't found the cause to).
2021-05-31T09:57:31Z
https://github.com/huggingface/datasets/pull/2370#issuecomment-851374623
https://api.github.com/repos/huggingface/datasets/issues/2370
MDEyOklzc3VlQ29tbWVudDg1MTM3NDYyMw==
https://api.github.com/repos/huggingface/datasets/issues/comments/851374623/reactions
2021-05-31T09:57:31Z
https://api.github.com/repos/huggingface/datasets/issues/comments/851374623
https://avatars.githubusercontent.com/u/43451571?v=4
https://api.github.com/users/andyzoujm/events{/privacy}
https://api.github.com/users/andyzoujm/followers
https://api.github.com/users/andyzoujm/following{/other_user}
https://api.github.com/users/andyzoujm/gists{/gist_id}
https://github.com/andyzoujm
andyzoujm
MDQ6VXNlcjQzNDUxNTcx
https://api.github.com/users/andyzoujm/orgs
https://api.github.com/users/andyzoujm/received_events
https://api.github.com/users/andyzoujm/repos
https://api.github.com/users/andyzoujm/starred{/owner}{/repo}
https://api.github.com/users/andyzoujm/subscriptions
User
https://api.github.com/users/andyzoujm
public
MEMBER
I took a look at the dummy data and some csv lines were cropped. I fixed them :)
2021-05-31T14:25:09Z
https://github.com/huggingface/datasets/pull/2370#issuecomment-851526051
https://api.github.com/repos/huggingface/datasets/issues/2370
MDEyOklzc3VlQ29tbWVudDg1MTUyNjA1MQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/851526051/reactions
2021-05-31T14:25:09Z
https://api.github.com/repos/huggingface/datasets/issues/comments/851526051
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
@andyzoujm Any reason why this dataset scrip was called "hendrycks_test" instead of "mmlu"?

We are thinking of renaming it...
2023-04-26T12:44:03Z
https://github.com/huggingface/datasets/pull/2370#issuecomment-1523358110
https://api.github.com/repos/huggingface/datasets/issues/2370
IC_kwDODunzps5azJme
https://api.github.com/repos/huggingface/datasets/issues/comments/1523358110/reactions
2023-04-26T12:44:03Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1523358110
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
CONTRIBUTOR
That's because we didn't call it MMLU in the paper (the shorthand didn't
emerge until over a year later), and people at OpenAI were calling it that.

Andy

On Wed, Apr 26, 2023 at 8:44 AM Albert Villanova del Moral <
***@***.***> wrote:

> @andyzoujm <https://github.com/andyzoujm> Any reason why this dataset
> scrip was called "hendrycks_test" instead of "mmlu"?
>
> We are thinking of renaming it...
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/datasets/pull/2370#issuecomment-1523358110>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AKLQJMZZFZBGJTBOIFWJ5KDXDEKB5ANCNFSM45BAOSIQ>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>

2023-04-27T19:18:39Z
https://github.com/huggingface/datasets/pull/2370#issuecomment-1526225681
https://api.github.com/repos/huggingface/datasets/issues/2370
IC_kwDODunzps5a-FsR
https://api.github.com/repos/huggingface/datasets/issues/comments/1526225681/reactions
2023-04-27T19:18:39Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1526225681
https://avatars.githubusercontent.com/u/43451571?v=4
https://api.github.com/users/andyzoujm/events{/privacy}
https://api.github.com/users/andyzoujm/followers
https://api.github.com/users/andyzoujm/following{/other_user}
https://api.github.com/users/andyzoujm/gists{/gist_id}
https://github.com/andyzoujm
andyzoujm
MDQ6VXNlcjQzNDUxNTcx
https://api.github.com/users/andyzoujm/orgs
https://api.github.com/users/andyzoujm/received_events
https://api.github.com/users/andyzoujm/repos
https://api.github.com/users/andyzoujm/starred{/owner}{/repo}
https://api.github.com/users/andyzoujm/subscriptions
User
https://api.github.com/users/andyzoujm
public
MEMBER
Thanks for your reply. Just for the records: we have renamed it to "cais/mmlu": https://huggingface.co/datasets/cais/mmlu
2023-05-11T05:42:56Z
https://github.com/huggingface/datasets/pull/2370#issuecomment-1543369725
https://api.github.com/repos/huggingface/datasets/issues/2370
IC_kwDODunzps5b_fP9
https://api.github.com/repos/huggingface/datasets/issues/comments/1543369725/reactions
2023-05-11T05:42:56Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1543369725
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
Adding Hendrycks test from https://arxiv.org/abs/2009.03300.
I'm having a bit of trouble with dummy data creation because some lines in the csv files aren't being loaded properly (only the first entry loaded in a row of length 6). The dataset is loading just fine. Hope you can kindly help!
Thank you!