fsspec 2022.8.2 breaks xopen in streaming mode
NONE
loading `fsspec==2022.7.1` fixes this issue, setup.py would need to be changed to prevent users from using the latest version of fsspec.
2022-09-09T17:29:00Z
https://github.com/huggingface/datasets/issues/4961#issuecomment-1242263786
https://api.github.com/repos/huggingface/datasets/issues/4961
IC_kwDODunzps5KC3Dq
https://api.github.com/repos/huggingface/datasets/issues/comments/1242263786/reactions
2022-09-09T17:29:00Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1242263786
https://avatars.githubusercontent.com/u/3616964?v=4
https://api.github.com/users/DCNemesis/events{/privacy}
https://api.github.com/users/DCNemesis/followers
https://api.github.com/users/DCNemesis/following{/other_user}
https://api.github.com/users/DCNemesis/gists{/gist_id}
https://github.com/DCNemesis
DCNemesis
MDQ6VXNlcjM2MTY5NjQ=
https://api.github.com/users/DCNemesis/orgs
https://api.github.com/users/DCNemesis/received_events
https://api.github.com/users/DCNemesis/repos
https://api.github.com/users/DCNemesis/starred{/owner}{/repo}
https://api.github.com/users/DCNemesis/subscriptions
User
https://api.github.com/users/DCNemesis
public
NONE
Opened [PR](https://github.com/huggingface/datasets/pull/4962) to address this.
2022-09-09T17:58:43Z
https://github.com/huggingface/datasets/issues/4961#issuecomment-1242301038
https://api.github.com/repos/huggingface/datasets/issues/4961
IC_kwDODunzps5KDAJu
https://api.github.com/repos/huggingface/datasets/issues/comments/1242301038/reactions
2022-09-09T17:58:43Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1242301038
https://avatars.githubusercontent.com/u/3616964?v=4
https://api.github.com/users/DCNemesis/events{/privacy}
https://api.github.com/users/DCNemesis/followers
https://api.github.com/users/DCNemesis/following{/other_user}
https://api.github.com/users/DCNemesis/gists{/gist_id}
https://github.com/DCNemesis
DCNemesis
MDQ6VXNlcjM2MTY5NjQ=
https://api.github.com/users/DCNemesis/orgs
https://api.github.com/users/DCNemesis/received_events
https://api.github.com/users/DCNemesis/repos
https://api.github.com/users/DCNemesis/starred{/owner}{/repo}
https://api.github.com/users/DCNemesis/subscriptions
User
https://api.github.com/users/DCNemesis
public
MEMBER
Hi @DCNemesis, thanks for reporting.

That was a temporary issue in `fsspec` releases 2022.8.0 and 2022.8.1. But they fixed it in their patch release 2022.8.2 (and yanked both previous versions). See:
- https://github.com/huggingface/transformers/pull/18846

Are you sure you have version 2022.8.2 installed?
```shell
pip install -U fsspec
```

2022-09-12T08:17:12Z
https://github.com/huggingface/datasets/issues/4961#issuecomment-1243376247
https://api.github.com/repos/huggingface/datasets/issues/4961
IC_kwDODunzps5KHGp3
https://api.github.com/repos/huggingface/datasets/issues/comments/1243376247/reactions
2022-09-12T08:18:25Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1243376247
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
NONE
@albertvillanova I was using a temporary Google Colab instance, but checking it again today it seems it was loading 2022.8.1 rather than 2022.8.2. It's surprising that colab is using the version that was replaced the same day it was released. Testing with 2022.8.2 did work. It appears Colab [will be fixing it](https://github.com/googlecolab/colabtools/issues/3055) on their end too. 
2022-09-12T11:55:53Z
https://github.com/huggingface/datasets/issues/4961#issuecomment-1243630024
https://api.github.com/repos/huggingface/datasets/issues/4961
IC_kwDODunzps5KIEnI
https://api.github.com/repos/huggingface/datasets/issues/comments/1243630024/reactions
2022-09-12T11:55:53Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1243630024
https://avatars.githubusercontent.com/u/3616964?v=4
https://api.github.com/users/DCNemesis/events{/privacy}
https://api.github.com/users/DCNemesis/followers
https://api.github.com/users/DCNemesis/following{/other_user}
https://api.github.com/users/DCNemesis/gists{/gist_id}
https://github.com/DCNemesis
DCNemesis
MDQ6VXNlcjM2MTY5NjQ=
https://api.github.com/users/DCNemesis/orgs
https://api.github.com/users/DCNemesis/received_events
https://api.github.com/users/DCNemesis/repos
https://api.github.com/users/DCNemesis/starred{/owner}{/repo}
https://api.github.com/users/DCNemesis/subscriptions
User
https://api.github.com/users/DCNemesis
public
MEMBER
Thanks for the additional information.

Once we know 2022.8.2 works, I'm closing this issue. Feel free to reopen it if necessary.
2022-09-12T14:32:05Z
https://github.com/huggingface/datasets/issues/4961#issuecomment-1243832734
https://api.github.com/repos/huggingface/datasets/issues/4961
IC_kwDODunzps5KI2Ge
https://api.github.com/repos/huggingface/datasets/issues/comments/1243832734/reactions
2022-09-12T14:32:05Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1243832734
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
MEMBER
Colab just upgraded their default `fsspec` version to 2022.8.2:
- https://github.com/googlecolab/colabtools/issues/3055#issuecomment-1244019010
2022-09-12T17:45:50Z
https://github.com/huggingface/datasets/issues/4961#issuecomment-1244087450
https://api.github.com/repos/huggingface/datasets/issues/4961
IC_kwDODunzps5KJ0Sa
https://api.github.com/repos/huggingface/datasets/issues/comments/1244087450/reactions
2022-09-12T17:45:50Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1244087450
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
## Describe the bug
When fsspec 2022.8.2 is installed in your environment, xopen will prematurely close files, making streaming mode inoperable.

## Steps to reproduce the bug
```python

import datasets

data = datasets.load_dataset('MLCommons/ml_spoken_words', 'id_wav', split='train', streaming=True)

```

## Expected results
Dataset should load as iterator.

## Actual results
```
[/usr/local/lib/python3.7/dist-packages/datasets/load.py](https://localhost:8080/#) in load_dataset(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, ignore_verifications, keep_in_memory, save_infos, revision, use_auth_token, task, streaming, **config_kwargs)
   1737     # Return iterable dataset in case of streaming
   1738     if streaming:
-> 1739         return builder_instance.as_streaming_dataset(split=split)
   1740 
   1741     # Some datasets are already processed on the HF google storage

[/usr/local/lib/python3.7/dist-packages/datasets/builder.py](https://localhost:8080/#) in as_streaming_dataset(self, split, base_path)
   1023         )
   1024         self._check_manual_download(dl_manager)
-> 1025         splits_generators = {sg.name: sg for sg in self._split_generators(dl_manager)}
   1026         # By default, return all splits
   1027         if split is None:

[~/.cache/huggingface/modules/datasets_modules/datasets/MLCommons--ml_spoken_words/321ea853cf0a05abb7a2d7efea900692a3d8622af65a2f3ce98adb7800a5d57b/ml_spoken_words.py](https://localhost:8080/#) in _split_generators(self, dl_manager)
    182                 name=datasets.Split.TRAIN,
    183                 gen_kwargs={
--> 184                     "audio_archives": [download_audio(split="train", lang=lang) for lang in self.config.languages],
    185                     "local_audio_archives_paths": [download_extract_audio(split="train", lang=lang) for lang in
    186                                                    self.config.languages] if not dl_manager.is_streaming else None,

[~/.cache/huggingface/modules/datasets_modules/datasets/MLCommons--ml_spoken_words/321ea853cf0a05abb7a2d7efea900692a3d8622af65a2f3ce98adb7800a5d57b/ml_spoken_words.py](https://localhost:8080/#) in <listcomp>(.0)
    182                 name=datasets.Split.TRAIN,
    183                 gen_kwargs={
--> 184                     "audio_archives": [download_audio(split="train", lang=lang) for lang in self.config.languages],
    185                     "local_audio_archives_paths": [download_extract_audio(split="train", lang=lang) for lang in
    186                                                    self.config.languages] if not dl_manager.is_streaming else None,

[~/.cache/huggingface/modules/datasets_modules/datasets/MLCommons--ml_spoken_words/321ea853cf0a05abb7a2d7efea900692a3d8622af65a2f3ce98adb7800a5d57b/ml_spoken_words.py](https://localhost:8080/#) in _download_audio_archives(dl_manager, lang, format, split)
    267 # for streaming case
    268 def _download_audio_archives(dl_manager, lang, format, split):
--> 269     archives_paths = _download_audio_archives_paths(dl_manager, lang, format, split)
    270     return [dl_manager.iter_archive(archive_path) for archive_path in archives_paths]

[~/.cache/huggingface/modules/datasets_modules/datasets/MLCommons--ml_spoken_words/321ea853cf0a05abb7a2d7efea900692a3d8622af65a2f3ce98adb7800a5d57b/ml_spoken_words.py](https://localhost:8080/#) in _download_audio_archives_paths(dl_manager, lang, format, split)
    251     n_files_path = dl_manager.download(n_files_url)
    252 
--> 253     with open(n_files_path, "r", encoding="utf-8") as file:
    254         n_files = int(file.read().strip())  # the file contains a number of archives
    255 

ValueError: I/O operation on closed file.
```


## Environment info
- `datasets` version: 2.4.0
- Platform: Linux-5.10.133+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.13
- PyArrow version: 6.0.1
- Pandas version: 1.3.5
