load_dataset using default cache on Windows causes PermissionError: [WinError 5] Access is denied
MEMBER
Hi @daqieq, thanks for reporting.

Unfortunately, I was not able to reproduce this bug:
```ipython
In [1]: from datasets import load_dataset
   ...: ds = load_dataset('wiki_bio')
Downloading: 7.58kB [00:00, 26.3kB/s]
Downloading: 2.71kB [00:00, ?B/s]
Using custom data configuration default
Downloading and preparing dataset wiki_bio/default (download: 318.53 MiB, generated: 736.94 MiB, post-processed: Unknown size, total: 1.03 GiB) to C:\Users\username\.cache\huggingface\datasets\wiki_bio\default\
1.1.0\5293ce565954ba965dada626f1e79684e98172d950371d266bf3caaf87e911c9...
Downloading: 334MB [01:17, 4.32MB/s]
Dataset wiki_bio downloaded and prepared to C:\Users\username\.cache\huggingface\datasets\wiki_bio\default\1.1.0\5293ce565954ba965dada626f1e79684e98172d950371d266bf3caaf87e911c9. Subsequent calls will reuse thi
s data.
```

This kind of error messages usually happen because:
- Your running Python script hasn't write access to that directory
- You have another program (the File Explorer?) already browsing inside that directory
2021-09-20T07:19:22Z
https://github.com/huggingface/datasets/issues/2937#issuecomment-922687865
https://api.github.com/repos/huggingface/datasets/issues/2937
IC_kwDODunzps42_xl5
https://api.github.com/repos/huggingface/datasets/issues/comments/922687865/reactions
2021-09-20T07:19:22Z
https://api.github.com/repos/huggingface/datasets/issues/comments/922687865
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
NONE
Thanks @albertvillanova for looking at it! I tried on my personal Windows machine and it downloaded just fine.

Running on my work machine and on a colleague's machine it is consistently hitting this error. It's not a write access issue because the `.incomplete` directory is written just fine. It just won't rename and then it deletes the directory in the `finally` step. Also the zip file is written and extracted fine in the downloads directory.

That leaves another program that might be interfering, and there are plenty of those in my work machine ... (full antivirus, data loss prevention, etc.). So the question remains, why not extend the `try` block to allow catching the error and circle back to the rename after the unknown program is finished doing its 'stuff'. This is the approach that I read about in the linked repo (see my comments above).

If it's not high priority, that's fine. However, if someone were to write an PR that solved this issue in our environment in an `except` clause, would it be reviewed for inclusion in a future release? Just wondering whether I should spend any more time on this issue.
2021-09-21T02:04:35Z
https://github.com/huggingface/datasets/issues/2937#issuecomment-923520704
https://api.github.com/repos/huggingface/datasets/issues/2937
IC_kwDODunzps43C87A
https://api.github.com/repos/huggingface/datasets/issues/comments/923520704/reactions
2021-09-21T02:04:35Z
https://api.github.com/repos/huggingface/datasets/issues/comments/923520704
https://avatars.githubusercontent.com/u/40532020?v=4
https://api.github.com/users/daqieq/events{/privacy}
https://api.github.com/users/daqieq/followers
https://api.github.com/users/daqieq/following{/other_user}
https://api.github.com/users/daqieq/gists{/gist_id}
https://github.com/daqieq
daqieq
MDQ6VXNlcjQwNTMyMDIw
https://api.github.com/users/daqieq/orgs
https://api.github.com/users/daqieq/received_events
https://api.github.com/users/daqieq/repos
https://api.github.com/users/daqieq/starred{/owner}{/repo}
https://api.github.com/users/daqieq/subscriptions
User
https://api.github.com/users/daqieq
public
CONTRIBUTOR
Hi @albertvillanova, even I am facing the same issue on my work machine:

`Downloading and preparing dataset json/c4-en-html-with-metadata to C:\Users\......\.cache\huggingface\datasets\json\c4-en-html-with-metadata-4635c2fd9249f62d\0.0.0\c90812beea906fcffe0d5e3bb9eba909a80a998b5f88e9f8acbd320aa91acfde...
100%|███████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 983.42it/s]
100%|███████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 209.01it/s]
Traceback (most recent call last):
  File "bsmetadata/preprocessing_utils.py", line 710, in <module>
    ds = load_dataset(
  File "C:\Users\.......\AppData\Roaming\Python\Python38\site-packages\datasets\load.py", line 1694, in load_dataset
    builder_instance.download_and_prepare(
  File "C:\Users\........\AppData\Roaming\Python\Python38\site-packages\datasets\builder.py", line 603, in download_and_prepare
    self._save_info()
  File "C:\Users\..........\AppData\Local\Programs\Python\Python38\lib\contextlib.py", line 120, in __exit__
    next(self.gen)
  File "C:\Users\.....\AppData\Roaming\Python\Python38\site-packages\datasets\builder.py", line 557, in incomplete_dir
    os.rename(tmp_dir, dirname)
PermissionError: [WinError 5] Access is denied: 'C:\\Users\\.........\\.cache\\huggingface\\datasets\\json\\c4-en-html-with-metadata-4635c2fd9249f62d\\0.0.0\\c90812beea906fcffe0d5e3bb9eba909a80a998b5f88e9f8acbd320aa91acfde.incomplete' -> 'C:\\Users\\I355109\\.cache\\huggingface\\datasets\\json\\c4-en-html-with-metadata-4635c2fd9249f62d\\0.0.0\\c90812beea906fcffe0d5e3bb9eba909a80a998b5f88e9f8acbd320aa91acfde'`
2022-01-29T03:49:30Z
https://github.com/huggingface/datasets/issues/2937#issuecomment-1024825163
https://api.github.com/repos/huggingface/datasets/issues/2937
IC_kwDODunzps49FZdL
https://api.github.com/repos/huggingface/datasets/issues/comments/1024825163/reactions
2022-01-29T03:49:30Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1024825163
https://avatars.githubusercontent.com/u/6687858?v=4
https://api.github.com/users/manandey/events{/privacy}
https://api.github.com/users/manandey/followers
https://api.github.com/users/manandey/following{/other_user}
https://api.github.com/users/manandey/gists{/gist_id}
https://github.com/manandey
manandey
MDQ6VXNlcjY2ODc4NTg=
https://api.github.com/users/manandey/orgs
https://api.github.com/users/manandey/received_events
https://api.github.com/users/manandey/repos
https://api.github.com/users/manandey/starred{/owner}{/repo}
https://api.github.com/users/manandey/subscriptions
User
https://api.github.com/users/manandey
public
CONTRIBUTOR
I'm facing the same issue.

## System Information

- OS Edition: Windows 10 21H1
- OS build: 19043.1826
- Python version: 3.10.6 (installed using `choco install python`)
- datasets: 2.4.0
- PyArrow: 6.0.1

## Troubleshooting steps

- Restart the computer, unfortunately doesn't work! 🌚
- Checked the permissions of `~./cache/...`, looks fine.
- Tested with a simple file operation using the `open()` function and writing a hello_world.txt, it works fine.
- Tested with a different `cache_dir` value on the `load_dataset()`, e.g. "./data"
- Tested different datasets: `conll2003`, `squad_v2`, and `wiki_bio`.
- Downgraded datasets from `2.4.0` to `2.1.0`, issue persists.
- Tested it on WSL (Ubuntu 20.04), and it works! 
- Python reinstallation, in the first time downloading `conll2003` works fine, but `squad` or `squad_v2` raises Access Denied.
  - After the system or VSCode restart, the issue comes back.

## Resolution

I fixed it by changing the following command:

https://github.com/huggingface/datasets/blob/68cffe30917a9abed68d28caf54b40c10f977602/src/datasets/builder.py#L666

for

```python
shutil.move(tmp_dir, dirname)
```
2022-08-10T02:12:35Z
https://github.com/huggingface/datasets/issues/2937#issuecomment-1210070428
https://api.github.com/repos/huggingface/datasets/issues/2937
IC_kwDODunzps5IIDWc
https://api.github.com/repos/huggingface/datasets/issues/comments/1210070428/reactions
2022-08-11T03:17:34Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1210070428
https://avatars.githubusercontent.com/u/8703022?v=4
https://api.github.com/users/DougTrajano/events{/privacy}
https://api.github.com/users/DougTrajano/followers
https://api.github.com/users/DougTrajano/following{/other_user}
https://api.github.com/users/DougTrajano/gists{/gist_id}
https://github.com/DougTrajano
DougTrajano
MDQ6VXNlcjg3MDMwMjI=
https://api.github.com/users/DougTrajano/orgs
https://api.github.com/users/DougTrajano/received_events
https://api.github.com/users/DougTrajano/repos
https://api.github.com/users/DougTrajano/starred{/owner}{/repo}
https://api.github.com/users/DougTrajano/subscriptions
User
https://api.github.com/users/DougTrajano
public
## Describe the bug
Standard process to download and load the wiki_bio dataset causes PermissionError in Windows 10 and 11.

## Steps to reproduce the bug
```python
from datasets import load_dataset
ds = load_dataset('wiki_bio')
```

## Expected results
It is expected that the dataset downloads without any errors.

## Actual results
PermissionError see trace below:
```
Using custom data configuration default
Downloading and preparing dataset wiki_bio/default (download: 318.53 MiB, generated: 736.94 MiB, post-processed: Unknown size, total: 1.03 GiB) to C:\Users\username\.cache\huggingface\datasets\wiki_bio\default\1.1.0\5293ce565954ba965dada626f1e79684e98172d950371d266bf3caaf87e911c9...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\username\.conda\envs\hf\lib\site-packages\datasets\load.py", line 1112, in load_dataset
    builder_instance.download_and_prepare(
  File "C:\Users\username\.conda\envs\hf\lib\site-packages\datasets\builder.py", line 644, in download_and_prepare
    self._save_info()
  File "C:\Users\username\.conda\envs\hf\lib\contextlib.py", line 120, in __exit__
    next(self.gen)
  File "C:\Users\username\.conda\envs\hf\lib\site-packages\datasets\builder.py", line 598, in incomplete_dir
    os.rename(tmp_dir, dirname)
PermissionError: [WinError 5] Access is denied: 'C:\\Users\\username\\.cache\\huggingface\\datasets\\wiki_bio\\default\\1.1.0\\5293ce565954ba965dada626f1e79684e98172d950371d266bf3caaf87e911c9.incomplete' -> 'C:\\Users\\username\\.cache\\huggingface\\datasets\\wiki_bio\\default\\1.1.0\\5293ce565954ba965dada626f1e79684e98172d950371d266bf3caaf87e911c9'
```
By commenting out the os.rename() [L604](https://github.com/huggingface/datasets/blob/master/src/datasets/builder.py#L604) and the shutil.rmtree() [L607](https://github.com/huggingface/datasets/blob/master/src/datasets/builder.py#L607) lines, in my virtual environment, I was able to get the load process to complete, rename the directory manually and then rerun the `load_dataset('wiki_bio')` to get what I needed.

It seems that os.rename() in the `incomplete_dir` content manager is the culprit. Here's another project [Conan](https://github.com/conan-io/conan/issues/6560) with similar issue with os.rename() if it helps debug this issue.

## Environment info
- `datasets` version: 1.12.1
- Platform: Windows-10-10.0.22449-SP0
- Python version: 3.8.12
- PyArrow version: 5.0.0
