wmt datasets fail to load
MEMBER
@stas00 Mea culpa... May I fix this tomorrow morning?
2021-03-03T19:41:29Z
https://github.com/huggingface/datasets/issues/1981#issuecomment-790004794
https://api.github.com/repos/huggingface/datasets/issues/1981
MDEyOklzc3VlQ29tbWVudDc5MDAwNDc5NA==
https://api.github.com/repos/huggingface/datasets/issues/comments/790004794/reactions
2021-03-03T19:41:40Z
https://api.github.com/repos/huggingface/datasets/issues/comments/790004794
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
CONTRIBUTOR
yes, of course, I reverted to the version before that and it works ;)

but since a new release was just made you will probably need to make a hotfix.

and add the wmt to the tests?
2021-03-03T19:42:52Z
https://github.com/huggingface/datasets/issues/1981#issuecomment-790005662
https://api.github.com/repos/huggingface/datasets/issues/1981
MDEyOklzc3VlQ29tbWVudDc5MDAwNTY2Mg==
https://api.github.com/repos/huggingface/datasets/issues/comments/790005662/reactions
2021-03-03T19:42:52Z
https://api.github.com/repos/huggingface/datasets/issues/comments/790005662
https://avatars.githubusercontent.com/u/10676103?v=4
https://api.github.com/users/stas00/events{/privacy}
https://api.github.com/users/stas00/followers
https://api.github.com/users/stas00/following{/other_user}
https://api.github.com/users/stas00/gists{/gist_id}
https://github.com/stas00
stas00
MDQ6VXNlcjEwNjc2MTAz
https://api.github.com/users/stas00/orgs
https://api.github.com/users/stas00/received_events
https://api.github.com/users/stas00/repos
https://api.github.com/users/stas00/starred{/owner}{/repo}
https://api.github.com/users/stas00/subscriptions
User
https://api.github.com/users/stas00
public
MEMBER
Sure, I will implement a regression test!
2021-03-03T19:45:05Z
https://github.com/huggingface/datasets/issues/1981#issuecomment-790006926
https://api.github.com/repos/huggingface/datasets/issues/1981
MDEyOklzc3VlQ29tbWVudDc5MDAwNjkyNg==
https://api.github.com/repos/huggingface/datasets/issues/comments/790006926/reactions
2021-03-03T19:45:05Z
https://api.github.com/repos/huggingface/datasets/issues/comments/790006926
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
MEMBER
@stas00 it is fixed. @lhoestq are you releasing the hot fix or would you prefer me to do it?
2021-03-03T20:23:05Z
https://github.com/huggingface/datasets/issues/1981#issuecomment-790030189
https://api.github.com/repos/huggingface/datasets/issues/1981
MDEyOklzc3VlQ29tbWVudDc5MDAzMDE4OQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/790030189/reactions
2021-03-03T20:23:05Z
https://api.github.com/repos/huggingface/datasets/issues/comments/790030189
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
MEMBER
I'll do a patch release for this issue early tomorrow.

And yes we absolutly need tests for the wmt datasets: The missing tests for wmt are an artifact from the early development of the lib but now we have tools to generate automatically the dummy data used for tests :)
2021-03-03T23:07:39Z
https://github.com/huggingface/datasets/issues/1981#issuecomment-790138112
https://api.github.com/repos/huggingface/datasets/issues/1981
MDEyOklzc3VlQ29tbWVudDc5MDEzODExMg==
https://api.github.com/repos/huggingface/datasets/issues/comments/790138112/reactions
2021-03-03T23:07:39Z
https://api.github.com/repos/huggingface/datasets/issues/comments/790138112
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
still facing the same issue or similar:
from datasets import load_dataset
wtm14_test = load_dataset('wmt14',"de-en",cache_dir='./datasets')

~.cache\huggingface\modules\datasets_modules\datasets\wmt14\43e717d978d2261502b0194999583acb874ba73b0f4aed0ada2889d1bb00f36e\wmt_utils.py in _split_generators(self, dl_manager)
758 # Extract manually downloaded files.
759 manual_files = dl_manager.extract(manual_paths_dict)
--> 760 extraction_map = dict(downloaded_files, **manual_files)
761
762 for language in self.config.language_pair:

TypeError: type object argument after ** must be a mapping, not list
2021-03-04T14:16:47Z
https://github.com/huggingface/datasets/issues/1981#issuecomment-790648987
https://api.github.com/repos/huggingface/datasets/issues/1981
MDEyOklzc3VlQ29tbWVudDc5MDY0ODk4Nw==
https://api.github.com/repos/huggingface/datasets/issues/comments/790648987/reactions
2021-03-04T14:16:47Z
https://api.github.com/repos/huggingface/datasets/issues/comments/790648987
https://avatars.githubusercontent.com/u/32322564?v=4
https://api.github.com/users/sabania/events{/privacy}
https://api.github.com/users/sabania/followers
https://api.github.com/users/sabania/following{/other_user}
https://api.github.com/users/sabania/gists{/gist_id}
https://github.com/sabania
sabania
MDQ6VXNlcjMyMzIyNTY0
https://api.github.com/users/sabania/orgs
https://api.github.com/users/sabania/received_events
https://api.github.com/users/sabania/repos
https://api.github.com/users/sabania/starred{/owner}{/repo}
https://api.github.com/users/sabania/subscriptions
User
https://api.github.com/users/sabania
public
on master:
```
python -c 'from datasets import load_dataset; load_dataset("wmt14", "de-en")'
Downloading and preparing dataset wmt14/de-en (download: Unknown size, generated: Unknown size, post-processed: Unknown size, total: Unknown size) to /home/stas/.cache/huggingface/datasets/wmt14/de-en/1.0.0/43e717d978d2261502b0194999583acb874ba73b0f4aed0ada2889d1bb00f36e...
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/mnt/nvme1/code/huggingface/datasets-master/src/datasets/load.py", line 740, in load_dataset
    builder_instance.download_and_prepare(
  File "/mnt/nvme1/code/huggingface/datasets-master/src/datasets/builder.py", line 578, in download_and_prepare
    self._download_and_prepare(
  File "/mnt/nvme1/code/huggingface/datasets-master/src/datasets/builder.py", line 634, in _download_and_prepare
    split_generators = self._split_generators(dl_manager, **split_generators_kwargs)
  File "/home/stas/.cache/huggingface/modules/datasets_modules/datasets/wmt14/43e717d978d2261502b0194999583acb874ba73b0f4aed0ada2889d1bb00f36e/wmt_utils.py", line 760, in _split_generators
    extraction_map = dict(downloaded_files, **manual_files)
```

it worked fine recently. same problem if I try wmt16.

git bisect points to this commit from Feb 25 as the culprit https://github.com/huggingface/datasets/commit/792f1d9bb1c5361908f73e2ef7f0181b2be409fa

@albertvillanova 