BIG-bench
CONTRIBUTOR
> It looks like the CI is failing on windows because our windows CI is unable to clone the bigbench repository (maybe it has to do with filenames that are longer than 256 characters, which windows don't like). Could the smaller installation of bigbench via pip solve this issue ?
> Otherwise we can see how to remove this limitation in our windows CI.

I'm not sure.
If it's git's fault that it can't handle the long filenames, it will possibly be resolved by the pip install. If it's an issue with windows not liking long filenames after it's installed, then it will not be resolved.
I don't have a windows computer to try it on, but I might be able to tweek this PR and do an experiment to find out. 
We're waiting for a quota increase for the pip install (https://github.com/pypa/pypi-support/issues/1782). It's been pending for 2-3 weeks, and I don't have an estimate for when it will be resolved. 

>Regarding the dummy data zip files, I think we can just keep datasets/bigbench/dummy/abstract_narrative_understanding/1.0.0/dummy_data.zip and remove all the other ones. We just require to have at least one dummy_data.zip file.

Sounds great. I will trim that down. 
2022-04-11T15:54:43Z
https://github.com/huggingface/datasets/pull/4125#issuecomment-1095236939
https://api.github.com/repos/huggingface/datasets/issues/4125
IC_kwDODunzps5BR_1L
https://api.github.com/repos/huggingface/datasets/issues/comments/1095236939/reactions
2022-04-11T15:55:08Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1095236939
https://avatars.githubusercontent.com/u/43357549?v=4
https://api.github.com/users/andersjohanandreassen/events{/privacy}
https://api.github.com/users/andersjohanandreassen/followers
https://api.github.com/users/andersjohanandreassen/following{/other_user}
https://api.github.com/users/andersjohanandreassen/gists{/gist_id}
https://github.com/andersjohanandreassen
andersjohanandreassen
MDQ6VXNlcjQzMzU3NTQ5
https://api.github.com/users/andersjohanandreassen/orgs
https://api.github.com/users/andersjohanandreassen/received_events
https://api.github.com/users/andersjohanandreassen/repos
https://api.github.com/users/andersjohanandreassen/starred{/owner}{/repo}
https://api.github.com/users/andersjohanandreassen/subscriptions
User
https://api.github.com/users/andersjohanandreassen
public
MEMBER
Do you know what are the other tests dependencies that have conflicts with bigbench ? I can try to split the CI to end up with a compatible list of test dependencies
2022-04-12T08:33:25Z
https://github.com/huggingface/datasets/pull/4125#issuecomment-1096359124
https://api.github.com/repos/huggingface/datasets/issues/4125
IC_kwDODunzps5BWRzU
https://api.github.com/repos/huggingface/datasets/issues/comments/1096359124/reactions
2022-04-12T08:33:25Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1096359124
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Hi @lhoestq,

I haven't played with eliminating requirements form the test dependencies, and I've been trying to resolve this by modifying the bigbench repo to become compatible. 
In the original bigbench repo, the version requirements were strict, and specifically it had a datasets==1.17.0 requirement which was causing trouble. 
I'm working on PR https://github.com/google/BIG-bench/pull/766 to get some more flexible versions that might be compatible with the test dependencies in  HF/datasets.
We're somewhat flexible in modifying these version numbers if we can figure out what the exact conflict is. 

I've spent some time experimenting with different versions, but I don't have a very efficient way of doing this debugging on my work computer (which for some reason doesn't produce the same sets of errors running python 3.9 instead of 3.6 or 3.7 in the tests). 
It currently fails at 
> The conflict is caused by:
>    bert-score 0.3.6 depends on matplotlib
>    big-bench 0.0.1 depends on matplotlib<4.0 and >=3.5.1

which doesn't seem like it can be the real issue. 

If you have any advice for how to resolve these conflicts, that would be greatly appreciated!
2022-04-12T15:33:54Z
https://github.com/huggingface/datasets/pull/4125#issuecomment-1096883064
https://api.github.com/repos/huggingface/datasets/issues/4125
IC_kwDODunzps5BYRt4
https://api.github.com/repos/huggingface/datasets/issues/comments/1096883064/reactions
2022-04-12T15:34:08Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1096883064
https://avatars.githubusercontent.com/u/43357549?v=4
https://api.github.com/users/andersjohanandreassen/events{/privacy}
https://api.github.com/users/andersjohanandreassen/followers
https://api.github.com/users/andersjohanandreassen/following{/other_user}
https://api.github.com/users/andersjohanandreassen/gists{/gist_id}
https://github.com/andersjohanandreassen
andersjohanandreassen
MDQ6VXNlcjQzMzU3NTQ5
https://api.github.com/users/andersjohanandreassen/orgs
https://api.github.com/users/andersjohanandreassen/received_events
https://api.github.com/users/andersjohanandreassen/repos
https://api.github.com/users/andersjohanandreassen/starred{/owner}{/repo}
https://api.github.com/users/andersjohanandreassen/subscriptions
User
https://api.github.com/users/andersjohanandreassen
public
CONTRIBUTOR
Hi again @lhoestq, 
After some more or less random guessing of conflicting packages, I've managed to find a configuration that seems to be compatible with HF/datasets. 

The errors went away after removing version limits on matplotlib and scipy, and loosening numpy from 1.19 -> 1.17 in the bigbench requirements. 

I might do some more tweaking to see if it lets me set some minimal limits on matplotlib and scipy, but I think we at least can move forward.

The WIN tests are still failing, now because of 

> Did not find path entry C:\tools\miniconda3\bin
>C:\tools\miniconda3\envs\py37\python.exe: No module named pytest

I have no way of debugging this locally, and unless there's some way to get more verbose logs, I don't know why it's not finding pytest. Would you be able to take a quick look? 

Update: Actually, I see it's still failing because of the long filenames. So perhaps the pytest error is just because the previous steps failed. 
2022-04-13T16:42:58Z
https://github.com/huggingface/datasets/pull/4125#issuecomment-1098266803
https://api.github.com/repos/huggingface/datasets/issues/4125
IC_kwDODunzps5Bdjiz
https://api.github.com/repos/huggingface/datasets/issues/comments/1098266803/reactions
2022-04-13T16:53:26Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1098266803
https://avatars.githubusercontent.com/u/43357549?v=4
https://api.github.com/users/andersjohanandreassen/events{/privacy}
https://api.github.com/users/andersjohanandreassen/followers
https://api.github.com/users/andersjohanandreassen/following{/other_user}
https://api.github.com/users/andersjohanandreassen/gists{/gist_id}
https://github.com/andersjohanandreassen
andersjohanandreassen
MDQ6VXNlcjQzMzU3NTQ5
https://api.github.com/users/andersjohanandreassen/orgs
https://api.github.com/users/andersjohanandreassen/received_events
https://api.github.com/users/andersjohanandreassen/repos
https://api.github.com/users/andersjohanandreassen/starred{/owner}{/repo}
https://api.github.com/users/andersjohanandreassen/subscriptions
User
https://api.github.com/users/andersjohanandreassen
public
CONTRIBUTOR
One more update on the WIN errors. 
I think all the long filenames are in files in the github repo that does not need to be included. 
We will try to remove them .
2022-04-13T18:16:07Z
https://github.com/huggingface/datasets/pull/4125#issuecomment-1098346237
https://api.github.com/repos/huggingface/datasets/issues/4125
IC_kwDODunzps5Bd279
https://api.github.com/repos/huggingface/datasets/issues/comments/1098346237/reactions
2022-04-13T18:16:07Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1098346237
https://avatars.githubusercontent.com/u/43357549?v=4
https://api.github.com/users/andersjohanandreassen/events{/privacy}
https://api.github.com/users/andersjohanandreassen/followers
https://api.github.com/users/andersjohanandreassen/following{/other_user}
https://api.github.com/users/andersjohanandreassen/gists{/gist_id}
https://github.com/andersjohanandreassen
andersjohanandreassen
MDQ6VXNlcjQzMzU3NTQ5
https://api.github.com/users/andersjohanandreassen/orgs
https://api.github.com/users/andersjohanandreassen/received_events
https://api.github.com/users/andersjohanandreassen/repos
https://api.github.com/users/andersjohanandreassen/starred{/owner}{/repo}
https://api.github.com/users/andersjohanandreassen/subscriptions
User
https://api.github.com/users/andersjohanandreassen
public
MEMBER
Hi ! The remaining error seems to be a `UnicodeDecodeError` from `setup.py`. I think you can fix your setup.py:
```diff
-     with open(os.path.join(os.path.dirname(__file__), fname)) as f:
+     with open(os.path.join(os.path.dirname(__file__), fname), encoding="utf-8") as f:
```
Indeed on windows, when you `open` a file it doesn't always use "utf-8" by default
2022-04-15T15:31:02Z
https://github.com/huggingface/datasets/pull/4125#issuecomment-1100178398
https://api.github.com/repos/huggingface/datasets/issues/4125
IC_kwDODunzps5Bk2Pe
https://api.github.com/repos/huggingface/datasets/issues/comments/1100178398/reactions
2022-04-15T15:31:25Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1100178398
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Hi @lhoestq, 
The dependency issues seems to now be resolved 🎉 

Now, the WIN tests are failing at
> ERROR tests/test_arrow_dataset.py::test_dummy_dataset_serialize_s3 - botocore...
> ERROR tests/test_dataset_dict.py::test_dummy_dataset_serialize_s3 - botocore...

Is this testing the dummy dataset that's added in bigbench? If so, I might need some help getting the right format in.

The error message I'm seeing is 
> raise EndpointConnectionError(endpoint_url=request.url, error=e)
> E           botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "http://127.0.0.1:5555/test"

Which seems unrelated, but perhaps the real issue is somewhere I'm not seeing? 
2022-04-20T18:11:51Z
https://github.com/huggingface/datasets/pull/4125#issuecomment-1104264717
https://api.github.com/repos/huggingface/datasets/issues/4125
IC_kwDODunzps5B0b4N
https://api.github.com/repos/huggingface/datasets/issues/comments/1104264717/reactions
2022-04-20T18:12:15Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1104264717
https://avatars.githubusercontent.com/u/43357549?v=4
https://api.github.com/users/andersjohanandreassen/events{/privacy}
https://api.github.com/users/andersjohanandreassen/followers
https://api.github.com/users/andersjohanandreassen/following{/other_user}
https://api.github.com/users/andersjohanandreassen/gists{/gist_id}
https://github.com/andersjohanandreassen
andersjohanandreassen
MDQ6VXNlcjQzMzU3NTQ5
https://api.github.com/users/andersjohanandreassen/orgs
https://api.github.com/users/andersjohanandreassen/received_events
https://api.github.com/users/andersjohanandreassen/repos
https://api.github.com/users/andersjohanandreassen/starred{/owner}{/repo}
https://api.github.com/users/andersjohanandreassen/subscriptions
User
https://api.github.com/users/andersjohanandreassen
public
MEMBER
Woohoo awesome !

Let me check the CI error
2022-04-21T10:34:01Z
https://github.com/huggingface/datasets/pull/4125#issuecomment-1105034173
https://api.github.com/repos/huggingface/datasets/issues/4125
IC_kwDODunzps5B3Xu9
https://api.github.com/repos/huggingface/datasets/issues/comments/1105034173/reactions
2022-04-21T10:34:09Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1105034173
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
Can you try to re-run the CI, just in case CircleCI messed up ?
2022-04-21T10:39:56Z
https://github.com/huggingface/datasets/pull/4125#issuecomment-1105041218
https://api.github.com/repos/huggingface/datasets/issues/4125
IC_kwDODunzps5B3ZdC
https://api.github.com/repos/huggingface/datasets/issues/comments/1105041218/reactions
2022-04-21T10:39:56Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1105041218
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Hi @lhoestq, 
Rerunning did not seem to solve the problem. 
The `test_dummy_dataset_serialize_s3` error still seems to remain.
2022-04-21T14:46:16Z
https://github.com/huggingface/datasets/pull/4125#issuecomment-1105319405
https://api.github.com/repos/huggingface/datasets/issues/4125
IC_kwDODunzps5B4dXt
https://api.github.com/repos/huggingface/datasets/issues/comments/1105319405/reactions
2022-04-21T15:06:01Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1105319405
https://avatars.githubusercontent.com/u/43357549?v=4
https://api.github.com/users/andersjohanandreassen/events{/privacy}
https://api.github.com/users/andersjohanandreassen/followers
https://api.github.com/users/andersjohanandreassen/following{/other_user}
https://api.github.com/users/andersjohanandreassen/gists{/gist_id}
https://github.com/andersjohanandreassen
andersjohanandreassen
MDQ6VXNlcjQzMzU3NTQ5
https://api.github.com/users/andersjohanandreassen/orgs
https://api.github.com/users/andersjohanandreassen/received_events
https://api.github.com/users/andersjohanandreassen/repos
https://api.github.com/users/andersjohanandreassen/starred{/owner}{/repo}
https://api.github.com/users/andersjohanandreassen/subscriptions
User
https://api.github.com/users/andersjohanandreassen
public
CONTRIBUTOR
Hi again @lhoestq, 
I'm not sure if this is informative or not in terms of debugging, but I deleted the dummy data and the errors for windows still fail and the others still pass. 
Do you have any idea what could be causing this error on windows?
2022-04-26T18:40:14Z
https://github.com/huggingface/datasets/pull/4125#issuecomment-1110129185
https://api.github.com/repos/huggingface/datasets/issues/4125
IC_kwDODunzps5CKzoh
https://api.github.com/repos/huggingface/datasets/issues/comments/1110129185/reactions
2022-04-26T18:40:14Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1110129185
https://avatars.githubusercontent.com/u/43357549?v=4
https://api.github.com/users/andersjohanandreassen/events{/privacy}
https://api.github.com/users/andersjohanandreassen/followers
https://api.github.com/users/andersjohanandreassen/following{/other_user}
https://api.github.com/users/andersjohanandreassen/gists{/gist_id}
https://github.com/andersjohanandreassen
andersjohanandreassen
MDQ6VXNlcjQzMzU3NTQ5
https://api.github.com/users/andersjohanandreassen/orgs
https://api.github.com/users/andersjohanandreassen/received_events
https://api.github.com/users/andersjohanandreassen/repos
https://api.github.com/users/andersjohanandreassen/starred{/owner}{/repo}
https://api.github.com/users/andersjohanandreassen/subscriptions
User
https://api.github.com/users/andersjohanandreassen
public
NONE
_The documentation is not available anymore as the PR was closed or merged._
2022-05-05T15:06:01Z
https://github.com/huggingface/datasets/pull/4125#issuecomment-1118669474
https://api.github.com/repos/huggingface/datasets/issues/4125
IC_kwDODunzps5CrYqi
https://api.github.com/repos/huggingface/datasets/issues/comments/1118669474/reactions
2022-06-08T17:41:44Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1118669474
https://avatars.githubusercontent.com/u/99929124?v=4
https://api.github.com/users/HuggingFaceDocBuilderDev/events{/privacy}
https://api.github.com/users/HuggingFaceDocBuilderDev/followers
https://api.github.com/users/HuggingFaceDocBuilderDev/following{/other_user}
https://api.github.com/users/HuggingFaceDocBuilderDev/gists{/gist_id}
https://github.com/HuggingFaceDocBuilderDev
HuggingFaceDocBuilderDev
U_kgDOBfTMJA
https://api.github.com/users/HuggingFaceDocBuilderDev/orgs
https://api.github.com/users/HuggingFaceDocBuilderDev/received_events
https://api.github.com/users/HuggingFaceDocBuilderDev/repos
https://api.github.com/users/HuggingFaceDocBuilderDev/starred{/owner}{/repo}
https://api.github.com/users/HuggingFaceDocBuilderDev/subscriptions
User
https://api.github.com/users/HuggingFaceDocBuilderDev
public
MEMBER
Now the last question: let's have the dataset under`google/bigbench` @andersjohanandreassen ?

I think it would be nicer, this way you and anyone in your team can update the dataset card whevener you want without going through a github PR. You just need to join the https://huggingface.co/google page using your google email :)
2022-05-06T14:26:39Z
https://github.com/huggingface/datasets/pull/4125#issuecomment-1119683009
https://api.github.com/repos/huggingface/datasets/issues/4125
IC_kwDODunzps5CvQHB
https://api.github.com/repos/huggingface/datasets/issues/comments/1119683009/reactions
2022-05-06T14:27:03Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1119683009
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Hi @lhoestq, 

Thank you so much for the help! I really appreciate it!!!

After some discussion with the other bigbench organizers, I think there is a slight preference for bigbench to not be under google/bigbench since this is a collaboration with researchers from many different institutions/organizations beyond Google. 

I see the drawback with the updates to the dataset card having to go through a PR, but hopefully that won't be very frequent. 

We're finalizing putting the bigbench api on pip, so once that's finalized I just need to update the setup.py with the correct dependency and I think we are ready to merge. 
2022-05-16T14:22:51Z
https://github.com/huggingface/datasets/pull/4125#issuecomment-1127741179
https://api.github.com/repos/huggingface/datasets/issues/4125
IC_kwDODunzps5DN_b7
https://api.github.com/repos/huggingface/datasets/issues/comments/1127741179/reactions
2022-05-16T14:22:51Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1127741179
https://avatars.githubusercontent.com/u/43357549?v=4
https://api.github.com/users/andersjohanandreassen/events{/privacy}
https://api.github.com/users/andersjohanandreassen/followers
https://api.github.com/users/andersjohanandreassen/following{/other_user}
https://api.github.com/users/andersjohanandreassen/gists{/gist_id}
https://github.com/andersjohanandreassen
andersjohanandreassen
MDQ6VXNlcjQzMzU3NTQ5
https://api.github.com/users/andersjohanandreassen/orgs
https://api.github.com/users/andersjohanandreassen/received_events
https://api.github.com/users/andersjohanandreassen/repos
https://api.github.com/users/andersjohanandreassen/starred{/owner}{/repo}
https://api.github.com/users/andersjohanandreassen/subscriptions
User
https://api.github.com/users/andersjohanandreassen
public
MEMBER
Ok perfect, thank you !
2022-05-16T15:13:46Z
https://github.com/huggingface/datasets/pull/4125#issuecomment-1127800458
https://api.github.com/repos/huggingface/datasets/issues/4125
IC_kwDODunzps5DON6K
https://api.github.com/repos/huggingface/datasets/issues/comments/1127800458/reactions
2022-05-16T15:13:46Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1127800458
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
I noticed that in the latest windows CI run it takes forever to install the dependencies, was there any change in the bigbench dependencies recently ?
2022-05-16T16:12:14Z
https://github.com/huggingface/datasets/pull/4125#issuecomment-1127865118
https://api.github.com/repos/huggingface/datasets/issues/4125
IC_kwDODunzps5DOdse
https://api.github.com/repos/huggingface/datasets/issues/comments/1127865118/reactions
2022-05-16T16:12:14Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1127865118
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
oh, sorry! I just did a double check on the dependencies, and it seems like there is at least one left that should have been removed.  There's also one new one added. 
Let me get those removed again. Will ping you here when it's updated. 
2022-05-16T16:59:38Z
https://github.com/huggingface/datasets/pull/4125#issuecomment-1127913039
https://api.github.com/repos/huggingface/datasets/issues/4125
IC_kwDODunzps5DOpZP
https://api.github.com/repos/huggingface/datasets/issues/comments/1127913039/reactions
2022-05-16T16:59:38Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1127913039
https://avatars.githubusercontent.com/u/43357549?v=4
https://api.github.com/users/andersjohanandreassen/events{/privacy}
https://api.github.com/users/andersjohanandreassen/followers
https://api.github.com/users/andersjohanandreassen/following{/other_user}
https://api.github.com/users/andersjohanandreassen/gists{/gist_id}
https://github.com/andersjohanandreassen
andersjohanandreassen
MDQ6VXNlcjQzMzU3NTQ5
https://api.github.com/users/andersjohanandreassen/orgs
https://api.github.com/users/andersjohanandreassen/received_events
https://api.github.com/users/andersjohanandreassen/repos
https://api.github.com/users/andersjohanandreassen/starred{/owner}{/repo}
https://api.github.com/users/andersjohanandreassen/subscriptions
User
https://api.github.com/users/andersjohanandreassen
public
MEMBER
It looks like there is a circular dependency in `bigbench` at https://storage.googleapis.com/public_research_data/bigbench/bigbench-0.0.1.tar.gz

```python
>>> import bigbench.api.util as bb_utils
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/circleci/.pyenv/versions/3.6.15/lib/python3.6/site-packages/bigbench/api/util.py", line 29, in <module>
    import bigbench.models.query_logging_model as query_logging_model
  File "/home/circleci/.pyenv/versions/3.6.15/lib/python3.6/site-packages/bigbench/models/query_logging_model.py", line 23, in <module>
    import bigbench.api.util as util
AttributeError: module 'bigbench.api' has no attribute 'util'
```
2022-05-30T16:11:59Z
https://github.com/huggingface/datasets/pull/4125#issuecomment-1141317635
https://api.github.com/repos/huggingface/datasets/issues/4125
IC_kwDODunzps5EByAD
https://api.github.com/repos/huggingface/datasets/issues/comments/1141317635/reactions
2022-05-30T16:11:59Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1141317635
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Hi @lhoestq , 
I think we are ready to merge! 

I have one minor question that I haven't been able to figure out: 
Is there a way to bypass the `verify_infos` from triggering? I have `max_examples` as an argument to allow for selecting a fixed subset of the datasets (some of the tasks have *very* many examples). But this is a variable that's not specified by the configs, so it raises an `NonMatchingSplitsSizesError`.
I wasn't able to work my way around this, but perhaps there is a way to bypass this that I'm not seeing?
If this cannot be done, I'm happy to ignore this for now.

Regarding pypi, we are working on a release there, but I'm told there is some issue that there is a problem regarding the upload, and we are not sure when it will be resolved, and it's not in my control. 
I think merging this PR with the GCS is a great idea, and I will open a new PR when the pypi version is ready. 
2022-06-03T13:31:31Z
https://github.com/huggingface/datasets/pull/4125#issuecomment-1145969212
https://api.github.com/repos/huggingface/datasets/issues/4125
IC_kwDODunzps5ETho8
https://api.github.com/repos/huggingface/datasets/issues/comments/1145969212/reactions
2022-06-03T13:31:31Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1145969212
https://avatars.githubusercontent.com/u/43357549?v=4
https://api.github.com/users/andersjohanandreassen/events{/privacy}
https://api.github.com/users/andersjohanandreassen/followers
https://api.github.com/users/andersjohanandreassen/following{/other_user}
https://api.github.com/users/andersjohanandreassen/gists{/gist_id}
https://github.com/andersjohanandreassen
andersjohanandreassen
MDQ6VXNlcjQzMzU3NTQ5
https://api.github.com/users/andersjohanandreassen/orgs
https://api.github.com/users/andersjohanandreassen/received_events
https://api.github.com/users/andersjohanandreassen/repos
https://api.github.com/users/andersjohanandreassen/starred{/owner}{/repo}
https://api.github.com/users/andersjohanandreassen/subscriptions
User
https://api.github.com/users/andersjohanandreassen
public
MEMBER
Cool ! Merging then :D

> Is there a way to bypass the verify_infos from triggering? I have max_examples as an argument to allow for selecting a fixed subset of the datasets (some of the tasks have very many examples). But this is a variable that's not specified by the configs, so it raises an NonMatchingSplitsSizesError.

This is a bug, I opened an issue [here](https://github.com/huggingface/datasets/issues/4462). It should be easy to fix :)
2022-06-08T17:32:19Z
https://github.com/huggingface/datasets/pull/4125#issuecomment-1150200545
https://api.github.com/repos/huggingface/datasets/issues/4125
IC_kwDODunzps5Ejqrh
https://api.github.com/repos/huggingface/datasets/issues/comments/1150200545/reactions
2022-06-08T17:32:19Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1150200545
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
The bigbench page is available here ! https://huggingface.co/datasets/bigbench

I think we can update the dataset viewer to install bigbench on it, but since this is production code I'd rather use the version on pypi for bigbench when it comes out
2022-06-08T17:50:33Z
https://github.com/huggingface/datasets/pull/4125#issuecomment-1150216926
https://api.github.com/repos/huggingface/datasets/issues/4125
IC_kwDODunzps5Ejure
https://api.github.com/repos/huggingface/datasets/issues/comments/1150216926/reactions
2022-06-08T17:57:48Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1150216926
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
This PR adds all BIG-bench json tasks to huggingface/datasets.  