cache can't cleaned or disabled
CONTRIBUTOR
Hey I have a similar problem and found a workaround using [temporary directories](https://docs.python.org/3/library/tempfile.html):

```python
from tempfile import TemporaryDirectory

with TemporaryDirectory() as cache_dir:
    data = load_dataset('json', data_files=save_local_path, split='train', cache_dir=cache_dir)
```

But I do agree that it would be more intuitive if `datasets` supported this directly. Especially `disable_caching` is confusing, since it basically doesn't disable caching.
2024-12-04T09:10:16Z
https://github.com/huggingface/datasets/issues/7260#issuecomment-2516635449
https://api.github.com/repos/huggingface/datasets/issues/7260
IC_kwDODunzps6WAM85
https://api.github.com/repos/huggingface/datasets/issues/comments/2516635449/reactions
2024-12-11T09:04:52Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2516635449
https://avatars.githubusercontent.com/u/24520725?v=4
https://api.github.com/users/baskrahmer/events{/privacy}
https://api.github.com/users/baskrahmer/followers
https://api.github.com/users/baskrahmer/following{/other_user}
https://api.github.com/users/baskrahmer/gists{/gist_id}
https://github.com/baskrahmer
baskrahmer
MDQ6VXNlcjI0NTIwNzI1
https://api.github.com/users/baskrahmer/orgs
https://api.github.com/users/baskrahmer/received_events
https://api.github.com/users/baskrahmer/repos
https://api.github.com/users/baskrahmer/starred{/owner}{/repo}
https://api.github.com/users/baskrahmer/subscriptions
User
https://api.github.com/users/baskrahmer
public
### Describe the bug

I tried following ways, the cache can't be disabled. 

I got 2T data, but I also got more than 2T cache file. I got pressure on storage. I need to diable cache or cleaned immediately after processed. Following ways are all not working, please give some help!

```python
from datasets import disable_caching
from transformers import AutoTokenizer
disable_caching()

tokenizer = AutoTokenizer.from_pretrained(args.tokenizer_path)
def tokenization_fn(examples):
    column_name = 'text' if 'text' in examples else 'data'
    tokenized_inputs = tokenizer(
        examples[column_name], return_special_tokens_mask=True, truncation=False,
        max_length=tokenizer.model_max_length
    )
    return tokenized_inputs

data = load_dataset('json', data_files=save_local_path, split='train', cache_dir=None)
data.cleanup_cache_files()
updated_dataset = data.map(tokenization_fn, load_from_cache_file=False)
updated_dataset .cleanup_cache_files()
```

### Expected behavior

no cache file generated

### Environment info

Ubuntu 20.04.6 LTS
datasets  3.0.2