load_dataset() cannot find dataset_info.json with multiple training runs in parallel
MEMBER
Hi ! It fails because the dataset is already being prepared by your first run. I'd encourage you to prepare your dataset before using it for multiple trainings.

You can also specify another cache directory by passing `cache_dir=` to `load_dataset()`.
2023-01-13T17:41:56Z
https://github.com/huggingface/datasets/issues/5412#issuecomment-1382182668
https://api.github.com/repos/huggingface/datasets/issues/5412
IC_kwDODunzps5SYm8M
https://api.github.com/repos/huggingface/datasets/issues/comments/1382182668/reactions
2023-01-13T17:41:56Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1382182668
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
Thank you! What do you mean by prepare it beforehand? I am unclear how to conduct dataset preparation outside of using the `load_dataset` function.
2023-01-15T21:42:20Z
https://github.com/huggingface/datasets/issues/5412#issuecomment-1383260287
https://api.github.com/repos/huggingface/datasets/issues/5412
IC_kwDODunzps5ScuB_
https://api.github.com/repos/huggingface/datasets/issues/comments/1383260287/reactions
2023-01-15T21:42:20Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1383260287
https://avatars.githubusercontent.com/u/7139344?v=4
https://api.github.com/users/mtoles/events{/privacy}
https://api.github.com/users/mtoles/followers
https://api.github.com/users/mtoles/following{/other_user}
https://api.github.com/users/mtoles/gists{/gist_id}
https://github.com/mtoles
mtoles
MDQ6VXNlcjcxMzkzNDQ=
https://api.github.com/users/mtoles/orgs
https://api.github.com/users/mtoles/received_events
https://api.github.com/users/mtoles/repos
https://api.github.com/users/mtoles/starred{/owner}{/repo}
https://api.github.com/users/mtoles/subscriptions
User
https://api.github.com/users/mtoles
public
MEMBER
You can have a separate script that does load_dataset + map + save_to_disk to save your prepared dataset somewhere. Then in your training script you can reload the dataset with load_from_disk
2023-01-16T13:29:17Z
https://github.com/huggingface/datasets/issues/5412#issuecomment-1384064342
https://api.github.com/repos/huggingface/datasets/issues/5412
IC_kwDODunzps5SfyVW
https://api.github.com/repos/huggingface/datasets/issues/comments/1384064342/reactions
2023-01-16T13:29:17Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1384064342
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
Thank you! I believe I was running additional map steps after loading, resulting in the cache conflict. 
2023-01-19T20:28:43Z
https://github.com/huggingface/datasets/issues/5412#issuecomment-1397558108
https://api.github.com/repos/huggingface/datasets/issues/5412
IC_kwDODunzps5TTQtc
https://api.github.com/repos/huggingface/datasets/issues/comments/1397558108/reactions
2023-01-19T20:28:43Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1397558108
https://avatars.githubusercontent.com/u/7139344?v=4
https://api.github.com/users/mtoles/events{/privacy}
https://api.github.com/users/mtoles/followers
https://api.github.com/users/mtoles/following{/other_user}
https://api.github.com/users/mtoles/gists{/gist_id}
https://github.com/mtoles
mtoles
MDQ6VXNlcjcxMzkzNDQ=
https://api.github.com/users/mtoles/orgs
https://api.github.com/users/mtoles/received_events
https://api.github.com/users/mtoles/repos
https://api.github.com/users/mtoles/starred{/owner}{/repo}
https://api.github.com/users/mtoles/subscriptions
User
https://api.github.com/users/mtoles
public
### Describe the bug

I have a custom local dataset in JSON form. I am trying to do multiple training runs in parallel. The first training run runs with no issue. However, when I start another run on another GPU, the following code throws this error.

If there is a workaround to ignore the cache I think that would solve my problem too.

I am using datasets version 2.8.0. 

### Steps to reproduce the bug

1. Start training run of GPU 0 loading dataset from
```
load_dataset(
    "json",
    data_files=tr_dataset_path,
    split=f"train",
    download_mode="force_redownload",
)
```
2. While GPU 0 is training, start an identical run on GPU 1. GPU 1 will produce the following error:

```
Traceback (most recent call last):
  File "/local-scratch1/data/mt/code/qq/train.py", line 198, in <module>
    main()
  File "/home/username/.local/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/username/.local/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/username/.local/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/username/.local/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/local-scratch1/data/mt/code/qq/train.py", line 113, in main
    load_dataset(
  File "/home/username/miniconda3/envs/qq3/lib/python3.8/site-packages/datasets/load.py", line 1734, in load_dataset
    builder_instance = load_dataset_builder(
  File "/home/username/miniconda3/envs/qq3/lib/python3.8/site-packages/datasets/load.py", line 1518, in load_dataset_builder
    builder_instance: DatasetBuilder = builder_cls(
  File "/home/username/miniconda3/envs/qq3/lib/python3.8/site-packages/datasets/builder.py", line 366, in __init__
    self.info = DatasetInfo.from_directory(self._cache_dir)
  File "/home/username/miniconda3/envs/qq3/lib/python3.8/site-packages/datasets/info.py", line 313, in from_directory
    with fs.open(path_join(dataset_info_dir, config.DATASET_INFO_FILENAME), "r", encoding="utf-8") as f:
  File "/home/username/miniconda3/envs/qq3/lib/python3.8/site-packages/fsspec/spec.py", line 1094, in open
    self.open(
  File "/home/username/miniconda3/envs/qq3/lib/python3.8/site-packages/fsspec/spec.py", line 1106, in open
    f = self._open(
  File "/home/username/miniconda3/envs/qq3/lib/python3.8/site-packages/fsspec/implementations/local.py", line 175, in _open
    return LocalFileOpener(path, mode, fs=self, **kwargs)
  File "/home/username/miniconda3/envs/qq3/lib/python3.8/site-packages/fsspec/implementations/local.py", line 273, in __init__
    self._open()
  File "/home/username/miniconda3/envs/qq3/lib/python3.8/site-packages/fsspec/implementations/local.py", line 278, in _open
    self.f = open(self.path, mode=self.mode)
FileNotFoundError: [Errno 2] No such file or directory: '/home/username/.cache/huggingface/datasets/json/default-43d06a4aedb25e6d/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/dataset_info.json'
```

### Expected behavior

Expected behavior: 2nd GPU training run should run the same as 1st GPU training run.

### Environment info

Copy-and-paste the text below in your GitHub issue.

- `datasets` version: 2.8.0
- Platform: Linux-5.4.0-120-generic-x86_64-with-glibc2.10
- Python version: 3.8.15
- PyArrow version: 9.0.0
- Pandas version: 1.5.2