 Python Programming Puzzles
CONTRIBUTOR
👀 @TalSchuster
2021-06-14T13:29:41Z
https://github.com/huggingface/datasets/issues/2499#issuecomment-860685791
https://api.github.com/repos/huggingface/datasets/issues/2499
MDEyOklzc3VlQ29tbWVudDg2MDY4NTc5MQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/860685791/reactions
2021-06-14T13:29:41Z
https://api.github.com/repos/huggingface/datasets/issues/comments/860685791
https://avatars.githubusercontent.com/u/16107619?v=4
https://api.github.com/users/VictorSanh/events{/privacy}
https://api.github.com/users/VictorSanh/followers
https://api.github.com/users/VictorSanh/following{/other_user}
https://api.github.com/users/VictorSanh/gists{/gist_id}
https://github.com/VictorSanh
VictorSanh
MDQ6VXNlcjE2MTA3NjE5
https://api.github.com/users/VictorSanh/orgs
https://api.github.com/users/VictorSanh/received_events
https://api.github.com/users/VictorSanh/repos
https://api.github.com/users/VictorSanh/starred{/owner}{/repo}
https://api.github.com/users/VictorSanh/subscriptions
User
https://api.github.com/users/VictorSanh
public
NONE
Thanks @VictorSanh!
There's also a [notebook](https://aka.ms/python_puzzles) and [demo](https://aka.ms/python_puzzles_study) available now to try out some of the puzzles
2021-06-15T18:14:14Z
https://github.com/huggingface/datasets/issues/2499#issuecomment-861727125
https://api.github.com/repos/huggingface/datasets/issues/2499
MDEyOklzc3VlQ29tbWVudDg2MTcyNzEyNQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/861727125/reactions
2021-06-15T18:14:14Z
https://api.github.com/repos/huggingface/datasets/issues/comments/861727125
https://avatars.githubusercontent.com/u/15113045?v=4
https://api.github.com/users/TalSchuster/events{/privacy}
https://api.github.com/users/TalSchuster/followers
https://api.github.com/users/TalSchuster/following{/other_user}
https://api.github.com/users/TalSchuster/gists{/gist_id}
https://github.com/TalSchuster
TalSchuster
MDQ6VXNlcjE1MTEzMDQ1
https://api.github.com/users/TalSchuster/orgs
https://api.github.com/users/TalSchuster/received_events
https://api.github.com/users/TalSchuster/repos
https://api.github.com/users/TalSchuster/starred{/owner}{/repo}
https://api.github.com/users/TalSchuster/subscriptions
User
https://api.github.com/users/TalSchuster
public
## Adding a Dataset
- **Name:** Python Programming Puzzles
- **Description:** Programming challenge called programming puzzles, as an objective and comprehensive evaluation of program synthesis
- **Paper:** https://arxiv.org/pdf/2106.05784.pdf
- **Data:** https://github.com/microsoft/PythonProgrammingPuzzles ([Scrolling through the data](https://github.com/microsoft/PythonProgrammingPuzzles/blob/main/problems/README.md))
- **Motivation:** Spans a large range of difficulty, problems, and domains. A useful resource for evaluation as we don't have a clear understanding of the abilities and skills of extremely large LMs.

Note: it's a growing dataset (contributions are welcome), so we'll need careful versioning for this dataset.

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).
