Fix the environment variable for huggingface cache
MEMBER
Hi ! yes now `datasets` uses `huggingface_hub` to download and cache files from the HF Hub so you need to use `HF_HOME` (or manually `HF_HUB_CACHE` and `HF_DATASETS_CACHE` if you want to separate HF Hub cached files and cached datasets Arrow files)

So in your change I guess it needs to be `HF_HOME` instead of `HF_CACHE` ?
2024-10-08T15:09:59Z
https://github.com/huggingface/datasets/pull/7200#issuecomment-2400122375
https://api.github.com/repos/huggingface/datasets/issues/7200
IC_kwDODunzps6PDvYH
https://api.github.com/repos/huggingface/datasets/issues/comments/2400122375/reactions
2024-10-08T15:09:59Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2400122375
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Thank you for your comment. You are right. I am sorry for my mistake, I fixed it.
2024-10-08T15:18:24Z
https://github.com/huggingface/datasets/pull/7200#issuecomment-2400145254
https://api.github.com/repos/huggingface/datasets/issues/7200
IC_kwDODunzps6PD09m
https://api.github.com/repos/huggingface/datasets/issues/comments/2400145254/reactions
2024-10-08T15:18:24Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2400145254
https://avatars.githubusercontent.com/u/989899?v=4
https://api.github.com/users/torotoki/events{/privacy}
https://api.github.com/users/torotoki/followers
https://api.github.com/users/torotoki/following{/other_user}
https://api.github.com/users/torotoki/gists{/gist_id}
https://github.com/torotoki
torotoki
MDQ6VXNlcjk4OTg5OQ==
https://api.github.com/users/torotoki/orgs
https://api.github.com/users/torotoki/received_events
https://api.github.com/users/torotoki/repos
https://api.github.com/users/torotoki/starred{/owner}{/repo}
https://api.github.com/users/torotoki/subscriptions
User
https://api.github.com/users/torotoki
public
NONE
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_7200). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
2024-10-08T15:47:09Z
https://github.com/huggingface/datasets/pull/7200#issuecomment-2400213009
https://api.github.com/repos/huggingface/datasets/issues/7200
IC_kwDODunzps6PEFgR
https://api.github.com/repos/huggingface/datasets/issues/comments/2400213009/reactions
2024-10-08T15:47:09Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2400213009
https://avatars.githubusercontent.com/u/99929124?v=4
https://api.github.com/users/HuggingFaceDocBuilderDev/events{/privacy}
https://api.github.com/users/HuggingFaceDocBuilderDev/followers
https://api.github.com/users/HuggingFaceDocBuilderDev/following{/other_user}
https://api.github.com/users/HuggingFaceDocBuilderDev/gists{/gist_id}
https://github.com/HuggingFaceDocBuilderDev
HuggingFaceDocBuilderDev
U_kgDOBfTMJA
https://api.github.com/users/HuggingFaceDocBuilderDev/orgs
https://api.github.com/users/HuggingFaceDocBuilderDev/received_events
https://api.github.com/users/HuggingFaceDocBuilderDev/repos
https://api.github.com/users/HuggingFaceDocBuilderDev/starred{/owner}{/repo}
https://api.github.com/users/HuggingFaceDocBuilderDev/subscriptions
User
https://api.github.com/users/HuggingFaceDocBuilderDev
public
NONE
I just had this issue, and needed to move the setting the env code in the python file to top, before the import of the lib 
ie. 
```python
import os
LOCAL_DISK_MOUNT = '/mnt/data'

os.environ['HF_HOME'] = f'{LOCAL_DISK_MOUNT}/hf_cache/'
os.environ['HF_DATASETS_CACHE'] = f'{LOCAL_DISK_MOUNT}/datasets/'

from datasets import load_dataset
from datasets import load_dataset_builder
from psutil._common import bytes2human


```
2024-10-30T23:06:09Z
https://github.com/huggingface/datasets/pull/7200#issuecomment-2448636134
https://api.github.com/repos/huggingface/datasets/issues/7200
IC_kwDODunzps6R8zjm
https://api.github.com/repos/huggingface/datasets/issues/comments/2448636134/reactions
2024-10-30T23:10:27Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2448636134
https://avatars.githubusercontent.com/u/8270088?v=4
https://api.github.com/users/yukiman76/events{/privacy}
https://api.github.com/users/yukiman76/followers
https://api.github.com/users/yukiman76/following{/other_user}
https://api.github.com/users/yukiman76/gists{/gist_id}
https://github.com/yukiman76
yukiman76
MDQ6VXNlcjgyNzAwODg=
https://api.github.com/users/yukiman76/orgs
https://api.github.com/users/yukiman76/received_events
https://api.github.com/users/yukiman76/repos
https://api.github.com/users/yukiman76/starred{/owner}{/repo}
https://api.github.com/users/yukiman76/subscriptions
User
https://api.github.com/users/yukiman76
public
Resolve  #6256. As far as I tested, `HF_DATASETS_CACHE` was ignored and I could not specify the cache directory at all except for the default one by this environment variable. `HF_HOME` has worked. Perhaps the recent change on file downloading by `huggingface_hub` could affect this bug.

In my testing, I could not specify the cache directory even by `load_dataset("dataset_name" cache_dir="...")`. It might be another issue. I also welcome any advice to solve this issue.