Unable to Use Custom Dataset Locally
MEMBER
Hi ! I opened a PR in your repo to fix this :)
https://huggingface.co/datasets/zpn/pubchem_selfies/discussions/7

basically you need to use `open` for streaming to work properly
2022-10-04T09:14:55Z
https://github.com/huggingface/datasets/issues/5060#issuecomment-1266645003
https://api.github.com/repos/huggingface/datasets/issues/5060
IC_kwDODunzps5Lf3gL
https://api.github.com/repos/huggingface/datasets/issues/comments/1266645003/reactions
2022-10-04T09:14:55Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1266645003
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Thank you so much for this! Naive question, is this a feature of `open` or have you all overloaded it to be able to read from a URL? Any links to code/documentation would be greatly appreciated, I'd love to learn more
2022-10-04T10:59:13Z
https://github.com/huggingface/datasets/issues/5060#issuecomment-1266772366
https://api.github.com/repos/huggingface/datasets/issues/5060
IC_kwDODunzps5LgWmO
https://api.github.com/repos/huggingface/datasets/issues/comments/1266772366/reactions
2022-10-04T10:59:13Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1266772366
https://avatars.githubusercontent.com/u/33707069?v=4
https://api.github.com/users/zanussbaum/events{/privacy}
https://api.github.com/users/zanussbaum/followers
https://api.github.com/users/zanussbaum/following{/other_user}
https://api.github.com/users/zanussbaum/gists{/gist_id}
https://github.com/zanussbaum
zanussbaum
MDQ6VXNlcjMzNzA3MDY5
https://api.github.com/users/zanussbaum/orgs
https://api.github.com/users/zanussbaum/received_events
https://api.github.com/users/zanussbaum/repos
https://api.github.com/users/zanussbaum/starred{/owner}{/repo}
https://api.github.com/users/zanussbaum/subscriptions
User
https://api.github.com/users/zanussbaum
public
MEMBER
`datasets` extends `open` in dataset scripts to work with URLs. The builtin `open` from python only works with local files.

You can find the extension here: https://github.com/huggingface/datasets/blob/6ad430ba0cdeeb601170f732d4bd977f5c04594d/src/datasets/download/streaming_download_manager.py#L435-L451

I think we can create a docs section dedicated to streaming to explain how this works
2022-10-05T13:57:53Z
https://github.com/huggingface/datasets/issues/5060#issuecomment-1268475083
https://api.github.com/repos/huggingface/datasets/issues/5060
IC_kwDODunzps5Lm2TL
https://api.github.com/repos/huggingface/datasets/issues/comments/1268475083/reactions
2022-10-05T13:57:53Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1268475083
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
Closing this one - feel free to reopen if you have more questions
2022-10-06T14:29:17Z
https://github.com/huggingface/datasets/issues/5060#issuecomment-1270159115
https://api.github.com/repos/huggingface/datasets/issues/5060
IC_kwDODunzps5LtRcL
https://api.github.com/repos/huggingface/datasets/issues/comments/1270159115/reactions
2022-10-06T14:29:17Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1270159115
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
## Describe the bug
I have uploaded a [dataset](https://huggingface.co/datasets/zpn/pubchem_selfies) and followed the instructions from the [dataset_loader](https://huggingface.co/docs/datasets/dataset_script#download-data-files-and-organize-splits) tutorial. In that tutorial, it says 
```
If the data files live in the same folder or repository of the dataset script, 
you can just pass the relative paths to the files instead of URLs.
```
Accordingly, I put the [relative path](https://huggingface.co/datasets/zpn/pubchem_selfies/blob/main/pubchem_selfies.py#L76) to the data to be used. I was able to test the dataset and generate the metadata locally with `datasets-cli test path/to/<your-dataset-loading-script> --save_infos --all_configs`

However, if I try to load the data using `load_dataset`, I get the following error
```
with gzip.open(filepath, mode="rt") as f:
  File "/usr/local/Cellar/python@3.9/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/gzip.py", line 58, in open
    binary_file = GzipFile(filename, gz_mode, compresslevel)
  File "/usr/local/Cellar/python@3.9/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/gzip.py", line 173, in __init__
    fileobj = self.myfileobj = builtins.open(filename, mode or 'rb')
FileNotFoundError: [Errno 2] No such file or directory: 'https://huggingface.co/datasets/zpn/pubchem_selfies/resolve/main/data/Compound_021000001_021500000/Compound_021000001_021500000_SELFIES.jsonl.gz'
```


## Steps to reproduce the bug
```python
>>> from datasets import load_dataset
>>> dataset = load_dataset("zpn/pubchem_selfies", streaming=True)
>>> t = dataset["train"]
>>> for item in t:
......     print(item)
......     break

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/zachnussbaum/env/lib/python3.9/site-packages/datasets/iterable_dataset.py", line 723, in __iter__
    for key, example in self._iter():
  File "/Users/zachnussbaum/env/lib/python3.9/site-packages/datasets/iterable_dataset.py", line 713, in _iter
    yield from ex_iterable
  File "/Users/zachnussbaum/env/lib/python3.9/site-packages/datasets/iterable_dataset.py", line 113, in __iter__
    yield from self.generate_examples_fn(**self.kwargs)
  File "/Users/zachnussbaum/.cache/huggingface/modules/datasets_modules/datasets/zpn--pubchem_selfies/d2571f35996765aea70fd3f3f8e3882d59c401fb738615c79282e2eb1d9f7a25/pubchem_selfies.py", line 475, in _generate_examples
    with gzip.open(filepath, mode="rt") as f:
  File "/usr/local/Cellar/python@3.9/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/gzip.py", line 58, in open
    binary_file = GzipFile(filename, gz_mode, compresslevel)
  File "/usr/local/Cellar/python@3.9/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/gzip.py", line 173, in __init__
    fileobj = self.myfileobj = builtins.open(filename, mode or 'rb')
FileNotFoundError: [Errno 2] No such file or directory: 'https://huggingface.co/datasets/zpn/pubchem_selfies/resolve/main/data/Compound_021000001_021500000/Compound_021000001_021500000_SELFIES.jsonl.gz'
````
```

## Expected results
A clear and concise description of the expected results.

## Actual results
Specify the actual results or traceback.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 2.5.1
- Platform: macOS-12.5.1-x86_64-i386-64bit
- Python version: 3.9.7
- PyArrow version: 9.0.0
- Pandas version: 1.5.0
