error downloading bigscience-catalogue-lm-data/lm_en_wiktionary_filtered
NONE
It would help a lot to be able to preview the dataset - I'd like to see if the pronunciations are in the dataset, eg. for ["word"](https://en.wiktionary.org/wiki/word),

Pronunciation
([Received Pronunciation](https://en.wikipedia.org/wiki/Received_Pronunciation)) [IPA](https://en.wiktionary.org/wiki/Wiktionary:International_Phonetic_Alphabet)([key](https://en.wiktionary.org/wiki/Appendix:English_pronunciation)): /wɜːd/
([General American](https://en.wikipedia.org/wiki/General_American)) [enPR](https://en.wiktionary.org/wiki/Appendix:English_pronunciation): wûrd, [IPA](https://en.wiktionary.org/wiki/Wiktionary:International_Phonetic_Alphabet)([key](https://en.wiktionary.org/wiki/Appendix:English_pronunciation)): /wɝd/
2022-05-02T20:35:53Z
https://github.com/huggingface/datasets/issues/4268#issuecomment-1115341081
https://api.github.com/repos/huggingface/datasets/issues/4268
IC_kwDODunzps5CesEZ
https://api.github.com/repos/huggingface/datasets/issues/comments/1115341081/reactions
2022-05-02T20:35:53Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1115341081
https://avatars.githubusercontent.com/u/102043285?v=4
https://api.github.com/users/i-am-neo/events{/privacy}
https://api.github.com/users/i-am-neo/followers
https://api.github.com/users/i-am-neo/following{/other_user}
https://api.github.com/users/i-am-neo/gists{/gist_id}
https://github.com/i-am-neo
i-am-neo
U_kgDOBhUOlQ
https://api.github.com/users/i-am-neo/orgs
https://api.github.com/users/i-am-neo/received_events
https://api.github.com/users/i-am-neo/repos
https://api.github.com/users/i-am-neo/starred{/owner}{/repo}
https://api.github.com/users/i-am-neo/subscriptions
User
https://api.github.com/users/i-am-neo
public
MEMBER
Hi @i-am-neo, thanks for reporting.

Normally this dataset should be private and not accessible for public use. @cakiki, @lvwerra, any reason why is it public? I see many other Wikimedia datasets are also public.

Also note that last commit "Add metadata" (https://huggingface.co/datasets/bigscience-catalogue-lm-data/lm_en_wiktionary_filtered/commit/dc2f458dab50e00f35c94efb3cd4009996858609) introduced buggy data files (`data/file-01.jsonl.gz.lock`, `data/file-01.jsonl.gz.lock.lock`). The same bug appears in other datasets as well.

@i-am-neo, please note that in the near future we are planning to make public all datasets used for the BigScience project (at least all of them whose license allows to do that). Once public, they will be accessible for all the NLP community.
2022-05-03T05:45:12Z
https://github.com/huggingface/datasets/issues/4268#issuecomment-1115757851
https://api.github.com/repos/huggingface/datasets/issues/4268
IC_kwDODunzps5CgR0b
https://api.github.com/repos/huggingface/datasets/issues/comments/1115757851/reactions
2022-05-03T05:45:12Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1115757851
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
CONTRIBUTOR
Ah this must be a bug introduced at creation time since the repos were created programmatically; I'll go ahead and make them private; sorry about that!
2022-05-03T07:59:39Z
https://github.com/huggingface/datasets/issues/4268#issuecomment-1115832018
https://api.github.com/repos/huggingface/datasets/issues/4268
IC_kwDODunzps5Cgj7S
https://api.github.com/repos/huggingface/datasets/issues/comments/1115832018/reactions
2022-05-03T07:59:39Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1115832018
https://avatars.githubusercontent.com/u/3664563?v=4
https://api.github.com/users/cakiki/events{/privacy}
https://api.github.com/users/cakiki/followers
https://api.github.com/users/cakiki/following{/other_user}
https://api.github.com/users/cakiki/gists{/gist_id}
https://github.com/cakiki
cakiki
MDQ6VXNlcjM2NjQ1NjM=
https://api.github.com/users/cakiki/orgs
https://api.github.com/users/cakiki/received_events
https://api.github.com/users/cakiki/repos
https://api.github.com/users/cakiki/starred{/owner}{/repo}
https://api.github.com/users/cakiki/subscriptions
User
https://api.github.com/users/cakiki
public
CONTRIBUTOR
All datasets are private now. 

Re:that bug I think we're currently avoiding it by avoiding verifications. (i.e. `ignore_verifications=True`)
2022-05-03T08:13:08Z
https://github.com/huggingface/datasets/issues/4268#issuecomment-1115841558
https://api.github.com/repos/huggingface/datasets/issues/4268
IC_kwDODunzps5CgmQW
https://api.github.com/repos/huggingface/datasets/issues/comments/1115841558/reactions
2022-05-03T08:13:08Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1115841558
https://avatars.githubusercontent.com/u/3664563?v=4
https://api.github.com/users/cakiki/events{/privacy}
https://api.github.com/users/cakiki/followers
https://api.github.com/users/cakiki/following{/other_user}
https://api.github.com/users/cakiki/gists{/gist_id}
https://github.com/cakiki
cakiki
MDQ6VXNlcjM2NjQ1NjM=
https://api.github.com/users/cakiki/orgs
https://api.github.com/users/cakiki/received_events
https://api.github.com/users/cakiki/repos
https://api.github.com/users/cakiki/starred{/owner}{/repo}
https://api.github.com/users/cakiki/subscriptions
User
https://api.github.com/users/cakiki
public
MEMBER
Thanks a lot, @cakiki.

@i-am-neo, I'm closing this issue for now because the dataset is not publicly available yet. Just stay tuned, as we will soon release all the BigScience open-license datasets.  
2022-05-03T11:23:48Z
https://github.com/huggingface/datasets/issues/4268#issuecomment-1115988925
https://api.github.com/repos/huggingface/datasets/issues/4268
IC_kwDODunzps5ChKO9
https://api.github.com/repos/huggingface/datasets/issues/comments/1115988925/reactions
2022-05-03T11:25:07Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1115988925
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
NONE
Thanks for letting me know, @albertvillanova @cakiki.
Any chance of having a subset alpha version in the meantime? 
I only need two dicts out of wiktionary: 1) phoneme(as key): word, and 2) word(as key): its phonemes.

Would like to use it for a mini-poc [Robust ASR](https://github.com/huggingface/transformers/issues/13162#issuecomment-1096881290) decoding, cc @patrickvonplaten. 

(Patrick, possible to email you so as not to litter github with comments? I have some observations after experiments training hubert on some YT AMI-like data (11.44% wer).  Also wonder if a robust ASR is on your/HG's roadmap).  Thanks!
2022-05-03T14:36:16Z
https://github.com/huggingface/datasets/issues/4268#issuecomment-1116176006
https://api.github.com/repos/huggingface/datasets/issues/4268
IC_kwDODunzps5Ch36G
https://api.github.com/repos/huggingface/datasets/issues/comments/1116176006/reactions
2022-05-03T14:36:16Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1116176006
https://avatars.githubusercontent.com/u/102043285?v=4
https://api.github.com/users/i-am-neo/events{/privacy}
https://api.github.com/users/i-am-neo/followers
https://api.github.com/users/i-am-neo/following{/other_user}
https://api.github.com/users/i-am-neo/gists{/gist_id}
https://github.com/i-am-neo
i-am-neo
U_kgDOBhUOlQ
https://api.github.com/users/i-am-neo/orgs
https://api.github.com/users/i-am-neo/received_events
https://api.github.com/users/i-am-neo/repos
https://api.github.com/users/i-am-neo/starred{/owner}{/repo}
https://api.github.com/users/i-am-neo/subscriptions
User
https://api.github.com/users/i-am-neo
public
CONTRIBUTOR
Hey @i-am-neo,

Cool to hear that you're working on Robust ASR! Feel free to drop me a mail :-)
2022-05-03T17:20:52Z
https://github.com/huggingface/datasets/issues/4268#issuecomment-1116351767
https://api.github.com/repos/huggingface/datasets/issues/4268
IC_kwDODunzps5Cii0X
https://api.github.com/repos/huggingface/datasets/issues/comments/1116351767/reactions
2022-05-03T17:20:52Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1116351767
https://avatars.githubusercontent.com/u/23423619?v=4
https://api.github.com/users/patrickvonplaten/events{/privacy}
https://api.github.com/users/patrickvonplaten/followers
https://api.github.com/users/patrickvonplaten/following{/other_user}
https://api.github.com/users/patrickvonplaten/gists{/gist_id}
https://github.com/patrickvonplaten
patrickvonplaten
MDQ6VXNlcjIzNDIzNjE5
https://api.github.com/users/patrickvonplaten/orgs
https://api.github.com/users/patrickvonplaten/received_events
https://api.github.com/users/patrickvonplaten/repos
https://api.github.com/users/patrickvonplaten/starred{/owner}{/repo}
https://api.github.com/users/patrickvonplaten/subscriptions
User
https://api.github.com/users/patrickvonplaten
public
CONTRIBUTOR
@i-am-neo This particular subset of the dataset was taken from the [CirrusSearch dumps](https://dumps.wikimedia.org/other/cirrussearch/current/)
You're specifically after the [enwiktionary-20220425-cirrussearch-content.json.gz](https://dumps.wikimedia.org/other/cirrussearch/current/enwiktionary-20220425-cirrussearch-content.json.gz) file
2022-05-04T11:30:27Z
https://github.com/huggingface/datasets/issues/4268#issuecomment-1117202865
https://api.github.com/repos/huggingface/datasets/issues/4268
IC_kwDODunzps5Clymx
https://api.github.com/repos/huggingface/datasets/issues/comments/1117202865/reactions
2022-05-04T11:30:27Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1117202865
https://avatars.githubusercontent.com/u/3664563?v=4
https://api.github.com/users/cakiki/events{/privacy}
https://api.github.com/users/cakiki/followers
https://api.github.com/users/cakiki/following{/other_user}
https://api.github.com/users/cakiki/gists{/gist_id}
https://github.com/cakiki
cakiki
MDQ6VXNlcjM2NjQ1NjM=
https://api.github.com/users/cakiki/orgs
https://api.github.com/users/cakiki/received_events
https://api.github.com/users/cakiki/repos
https://api.github.com/users/cakiki/starred{/owner}{/repo}
https://api.github.com/users/cakiki/subscriptions
User
https://api.github.com/users/cakiki
public
NONE
thanks @cakiki !  <del>I could access the gz file yesterday (but neglected to tuck it away somewhere safe), and today the link is throwing a 404. Can you help? </del>  Never mind, got it!
2022-05-06T15:05:17Z
https://github.com/huggingface/datasets/issues/4268#issuecomment-1119719741
https://api.github.com/repos/huggingface/datasets/issues/4268
IC_kwDODunzps5CvZE9
https://api.github.com/repos/huggingface/datasets/issues/comments/1119719741/reactions
2022-05-06T15:53:30Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1119719741
https://avatars.githubusercontent.com/u/102043285?v=4
https://api.github.com/users/i-am-neo/events{/privacy}
https://api.github.com/users/i-am-neo/followers
https://api.github.com/users/i-am-neo/following{/other_user}
https://api.github.com/users/i-am-neo/gists{/gist_id}
https://github.com/i-am-neo
i-am-neo
U_kgDOBhUOlQ
https://api.github.com/users/i-am-neo/orgs
https://api.github.com/users/i-am-neo/received_events
https://api.github.com/users/i-am-neo/repos
https://api.github.com/users/i-am-neo/starred{/owner}{/repo}
https://api.github.com/users/i-am-neo/subscriptions
User
https://api.github.com/users/i-am-neo
public
NONE
thanks @patrickvonplaten.  will do - getting my observations together.
2022-05-06T15:07:01Z
https://github.com/huggingface/datasets/issues/4268#issuecomment-1119721432
https://api.github.com/repos/huggingface/datasets/issues/4268
IC_kwDODunzps5CvZfY
https://api.github.com/repos/huggingface/datasets/issues/comments/1119721432/reactions
2022-05-06T15:07:01Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1119721432
https://avatars.githubusercontent.com/u/102043285?v=4
https://api.github.com/users/i-am-neo/events{/privacy}
https://api.github.com/users/i-am-neo/followers
https://api.github.com/users/i-am-neo/following{/other_user}
https://api.github.com/users/i-am-neo/gists{/gist_id}
https://github.com/i-am-neo
i-am-neo
U_kgDOBhUOlQ
https://api.github.com/users/i-am-neo/orgs
https://api.github.com/users/i-am-neo/received_events
https://api.github.com/users/i-am-neo/repos
https://api.github.com/users/i-am-neo/starred{/owner}{/repo}
https://api.github.com/users/i-am-neo/subscriptions
User
https://api.github.com/users/i-am-neo
public
## Describe the bug
Error generated when attempting to download dataset

## Steps to reproduce the bug
```python
from datasets import load_dataset

dataset = load_dataset("bigscience-catalogue-lm-data/lm_en_wiktionary_filtered")
```

## Expected results
A clear and concise description of the expected results.

## Actual results
```
ExpectedMoreDownloadedFiles               Traceback (most recent call last)

[<ipython-input-62-4ac5cf959477>](https://localhost:8080/#) in <module>()
      1 from datasets import load_dataset
      2 
----> 3 dataset = load_dataset("bigscience-catalogue-lm-data/lm_en_wiktionary_filtered")

3 frames

[/usr/local/lib/python3.7/dist-packages/datasets/utils/info_utils.py](https://localhost:8080/#) in verify_checksums(expected_checksums, recorded_checksums, verification_name)
     31         return
     32     if len(set(expected_checksums) - set(recorded_checksums)) > 0:
---> 33         raise ExpectedMoreDownloadedFiles(str(set(expected_checksums) - set(recorded_checksums)))
     34     if len(set(recorded_checksums) - set(expected_checksums)) > 0:
     35         raise UnexpectedDownloadedFile(str(set(recorded_checksums) - set(expected_checksums)))

ExpectedMoreDownloadedFiles: {'/home/leandro/catalogue_data/datasets/lm_en_wiktionary_filtered/data/file-01.jsonl.gz', '/home/leandro/catalogue_data/datasets/lm_en_wiktionary_filtered/data/file-01.jsonl.gz.lock'}
```


## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.18.3
- Platform: Linux-5.4.188+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.13
- PyArrow version: 6.0.1
