Force to reuse cache at given path
NONE
realized that need to pass the path at `cache_file_name` like

```python
tokenized_datasets = raw_datasets["train"].map(
                tokenize_function,
                batched=True,
                num_proc=data_args.preprocessing_num_workers,
                remove_columns=[text_column_name],
                load_from_cache_file=True,
                desc="Running tokenizer on dataset line_by_line",
                # cache_file_names= {"train": "cache-1982fea76aa54a13.arrow"}
                cache_file_name="/project/huggingface_cache/datasets/..../cache-1982fea76aa54a13.arrow",
                new_fingerprint="1982fea76aa54a13"
            )
```
2023-08-30T19:00:45Z
https://github.com/huggingface/datasets/issues/6195#issuecomment-1699688324
https://api.github.com/repos/huggingface/datasets/issues/6195
IC_kwDODunzps5lTy-E
https://api.github.com/repos/huggingface/datasets/issues/comments/1699688324/reactions
2023-08-30T19:00:45Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1699688324
https://avatars.githubusercontent.com/u/43507393?v=4
https://api.github.com/users/Luosuu/events{/privacy}
https://api.github.com/users/Luosuu/followers
https://api.github.com/users/Luosuu/following{/other_user}
https://api.github.com/users/Luosuu/gists{/gist_id}
https://github.com/Luosuu
Luosuu
MDQ6VXNlcjQzNTA3Mzkz
https://api.github.com/users/Luosuu/orgs
https://api.github.com/users/Luosuu/received_events
https://api.github.com/users/Luosuu/repos
https://api.github.com/users/Luosuu/starred{/owner}{/repo}
https://api.github.com/users/Luosuu/subscriptions
User
https://api.github.com/users/Luosuu
public
NONE
Thank you so much! I went through a lot of issues before finding similar experiences here. I have to say that the [docs](https://huggingface.co/docs/datasets/v2.11.0/en/package_reference/main_classes#datasets.Dataset.map) of `.map()` is really misleading, probably making people think that just assigning the file name to cache_file_name is enough.
2023-11-03T10:14:20Z
https://github.com/huggingface/datasets/issues/6195#issuecomment-1792173391
https://api.github.com/repos/huggingface/datasets/issues/6195
IC_kwDODunzps5q0mVP
https://api.github.com/repos/huggingface/datasets/issues/comments/1792173391/reactions
2023-11-03T10:14:20Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1792173391
https://avatars.githubusercontent.com/u/33591507?v=4
https://api.github.com/users/ZhouqyCH/events{/privacy}
https://api.github.com/users/ZhouqyCH/followers
https://api.github.com/users/ZhouqyCH/following{/other_user}
https://api.github.com/users/ZhouqyCH/gists{/gist_id}
https://github.com/ZhouqyCH
ZhouqyCH
MDQ6VXNlcjMzNTkxNTA3
https://api.github.com/users/ZhouqyCH/orgs
https://api.github.com/users/ZhouqyCH/received_events
https://api.github.com/users/ZhouqyCH/repos
https://api.github.com/users/ZhouqyCH/starred{/owner}{/repo}
https://api.github.com/users/ZhouqyCH/subscriptions
User
https://api.github.com/users/ZhouqyCH
public
### Describe the bug

I have run the official example of MLM like:

```bash
  python run_mlm.py \
      --model_name_or_path roberta-base \
      --dataset_name togethercomputer/RedPajama-Data-1T \
      --dataset_config_name arxiv \
      --per_device_train_batch_size 10 \
      --preprocessing_num_workers 20 \
      --validation_split_percentage 0 \
      --cache_dir /project/huggingface_cache/datasets \
      --line_by_line \
      --do_train \
      --pad_to_max_length \
      --output_dir /project/huggingface_cache/test-mlm
```
it successfully runs and at my cache folder has `cache-1982fea76aa54a13_00001_of_00020.arrow`.....  `cache-1982fea76aa54a13_00020_of_00020.arrow ` as tokenization cache of `map` method. And the cache works fine every time I run the command above.

However, when I switched to jupyter notebook (since I do not want to load datasets every time when I changed other parameters not related to the dataloading). It is not recognizing the cache files and starts to re-run the entire tokenization process. 

I changed my code to 
```python
tokenized_datasets = raw_datasets["train"].map(
                tokenize_function,
                batched=True,
                num_proc=data_args.preprocessing_num_workers,
                remove_columns=[text_column_name],
                load_from_cache_file=True,
                desc="Running tokenizer on dataset line_by_line",
                # cache_file_names= {"train": "cache-1982fea76aa54a13.arrow"}
                cache_file_name="cache-1982fea76aa54a13.arrow",
                new_fingerprint="1982fea76aa54a13"
            )
```
it still does not recognize the previously cached files and trying to re-run the tokenization process.

### Steps to reproduce the bug

use jupyter notebook for dataset map function.

### Expected behavior

the map function accepts the given cache_file_name and new_fingerprint then load the previously cached files.

### Environment info

- `datasets` version: 2.14.4.dev0
- Platform: Linux-3.10.0-1160.59.1.el7.x86_64-x86_64-with-glibc2.10
- Python version: 3.8.8
- Huggingface_hub version: 0.16.4
- PyArrow version: 12.0.1
- Pandas version: 2.0.3