Specifying an exisiting folder in download_and_prepare deletes everything in it
CONTRIBUTOR
cc @lhoestq 
2022-10-12T11:53:45Z
https://github.com/huggingface/datasets/issues/5105#issuecomment-1276037071
https://api.github.com/repos/huggingface/datasets/issues/5105
IC_kwDODunzps5MDsfP
https://api.github.com/repos/huggingface/datasets/issues/comments/1276037071/reactions
2022-10-12T11:53:45Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1276037071
https://avatars.githubusercontent.com/u/3664563?v=4
https://api.github.com/users/cakiki/events{/privacy}
https://api.github.com/users/cakiki/followers
https://api.github.com/users/cakiki/following{/other_user}
https://api.github.com/users/cakiki/gists{/gist_id}
https://github.com/cakiki
cakiki
MDQ6VXNlcjM2NjQ1NjM=
https://api.github.com/users/cakiki/orgs
https://api.github.com/users/cakiki/received_events
https://api.github.com/users/cakiki/repos
https://api.github.com/users/cakiki/starred{/owner}{/repo}
https://api.github.com/users/cakiki/subscriptions
User
https://api.github.com/users/cakiki
public
MEMBER
Thanks for reporting, @cakiki.

I would say the deletion of the dir is an expected behavior though...
2022-10-17T08:46:56Z
https://github.com/huggingface/datasets/issues/5105#issuecomment-1280504176
https://api.github.com/repos/huggingface/datasets/issues/5105
IC_kwDODunzps5MUvFw
https://api.github.com/repos/huggingface/datasets/issues/comments/1280504176/reactions
2022-10-17T08:47:45Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1280504176
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
MEMBER
`dask.to_parquet` has an "overwrite" parameter and default is `False`, we could also have something similar
2022-10-17T09:32:00Z
https://github.com/huggingface/datasets/issues/5105#issuecomment-1280563942
https://api.github.com/repos/huggingface/datasets/issues/5105
IC_kwDODunzps5MU9rm
https://api.github.com/repos/huggingface/datasets/issues/comments/1280563942/reactions
2022-10-17T09:32:00Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1280563942
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Thank you both for your feedback!

@albertvillanova I think I might have have the wrong mental model of what the function was meant to do. I thought it would be an API similar to the pandas `to_XX` write methods (Like the one @lhoestq mentions) so I just assumed it would download the dataframe to whichever folder I specififed (`"./"` in my case) so I could load it into a dask dataframe. I absolutely did not expect it to delete everything in my local directory, including the script where I called it from :smile: 

I think Quentin's proposed solution sounds like a reasonable feature!
2022-10-17T12:37:12Z
https://github.com/huggingface/datasets/issues/5105#issuecomment-1280791599
https://api.github.com/repos/huggingface/datasets/issues/5105
IC_kwDODunzps5MV1Qv
https://api.github.com/repos/huggingface/datasets/issues/comments/1280791599/reactions
2022-10-17T12:37:12Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1280791599
https://avatars.githubusercontent.com/u/3664563?v=4
https://api.github.com/users/cakiki/events{/privacy}
https://api.github.com/users/cakiki/followers
https://api.github.com/users/cakiki/following{/other_user}
https://api.github.com/users/cakiki/gists{/gist_id}
https://github.com/cakiki
cakiki
MDQ6VXNlcjM2NjQ1NjM=
https://api.github.com/users/cakiki/orgs
https://api.github.com/users/cakiki/received_events
https://api.github.com/users/cakiki/repos
https://api.github.com/users/cakiki/starred{/owner}{/repo}
https://api.github.com/users/cakiki/subscriptions
User
https://api.github.com/users/cakiki
public
MEMBER
actually there's already a `download_mode` parameter that defaults to `REUSE_DATASET_IF_EXISTS` - so I guess it's just a matter of not deleting files unrelated to the dataset, and to overwrite existing dataset files if the download mode is `REUSE_CACHE_IF_EXISTS` or `FORCE_REDOWNLOAD`
2022-10-20T11:53:59Z
https://github.com/huggingface/datasets/issues/5105#issuecomment-1285396212
https://api.github.com/repos/huggingface/datasets/issues/5105
IC_kwDODunzps5MnZb0
https://api.github.com/repos/huggingface/datasets/issues/comments/1285396212/reactions
2022-10-20T11:53:59Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1285396212
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
## Describe the bug
The builder correctly creates the `output_dir` folder if it doesn't exist, but if the folder exists everything within it is deleted. Specifying `"."` as the `output_dir` deletes everything in your current dir but also leads to **another bug** whose traceback is the following:

```
                          Traceback (most recent call last)
Input In [11], in <cell line: 1>()
----> 1 rotten_tomatoes_builder.download_and_prepare(output_dir=".", max_shard_size="200MB", file_format="parquet")

File ~/BIGSCIENCE/env/lib/python3.9/site-packages/datasets/builder.py:818, in download_and_prepare(self, output_dir, download_config, download_mode, ignore_verifications, try_from_hf_gcs, dl_manager, base_path, use_auth_token, file_format, max_shard_size, storage_options, **download_and_prepare_kwargs)

File /usr/lib/python3.9/contextlib.py:124, in _GeneratorContextManager.__exit__(self, type, value, traceback)
    122 if type is None:
    123     try:
--> 124         next(self.gen)
    125     except StopIteration:
    126         return False

File ~/BIGSCIENCE/env/lib/python3.9/site-packages/datasets/builder.py:760, in incomplete_dir(dirname)

File /usr/lib/python3.9/shutil.py:722, in rmtree(path, ignore_errors, onerror)
    720         os.rmdir(path)
    721     except OSError:
--> 722         onerror(os.rmdir, path, sys.exc_info())
    723 else:
    724     try:
    725         # symlinks to directories are forbidden, see bug #1669

File /usr/lib/python3.9/shutil.py:720, in rmtree(path, ignore_errors, onerror)
    718 _rmtree_safe_fd(fd, path, onerror)
    719 try:
--> 720     os.rmdir(path)
    721 except OSError:
    722     onerror(os.rmdir, path, sys.exc_info())

OSError: [Errno 22] Invalid argument: '/home/christopher/BIGSCIENCE/.'
``` 

## Steps to reproduce the bug
```python
rotten_tomatoes_builder = load_dataset_builder("rotten_tomatoes")
rotten_tomatoes_builder.download_and_prepare(output_dir="./test_folder", max_shard_size="200MB", file_format="parquet")
```
If `test_folder` contains any files they will all be deleted
 
## Expected results
Either a warning that all files will be deleted, but preferably that they not be deleted at all.

## Actual results
N/A

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 2.3.2
- Platform: Linux-5.15.0-48-generic-x86_64-with-glibc2.29
- Python version: 3.8.10
- PyArrow version: 8.0.0
- Pandas version: 1.4.3
