Cache is not transportable
MEMBER
Hi ! No the cache is not transportable in general. It will work on a shared filesystem if you use the same python environment, but not across machines/os/environments.

In particular, reloading cached datasets does work, but reloading cached processed datasets (e.g. from `map`) may not work. This is because some hashes used by caching are based on pickle dumps of the function you pass to `map`.

Finally you may copy the cache to another machine, but all the `cached-*.arrow` files are unlikely to be reloaded.
2023-02-28T11:35:58Z
https://github.com/huggingface/datasets/issues/5585#issuecomment-1448029806
https://api.github.com/repos/huggingface/datasets/issues/5585
IC_kwDODunzps5WTy5u
https://api.github.com/repos/huggingface/datasets/issues/comments/1448029806/reactions
2023-02-28T11:37:25Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1448029806
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
OK good to know. Thanks @lhoestq !
2023-02-28T21:26:52Z
https://github.com/huggingface/datasets/issues/5585#issuecomment-1448938609
https://api.github.com/repos/huggingface/datasets/issues/5585
IC_kwDODunzps5WXQxx
https://api.github.com/repos/huggingface/datasets/issues/comments/1448938609/reactions
2023-02-28T21:26:52Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1448938609
https://avatars.githubusercontent.com/u/4443482?v=4
https://api.github.com/users/davidgilbertson/events{/privacy}
https://api.github.com/users/davidgilbertson/followers
https://api.github.com/users/davidgilbertson/following{/other_user}
https://api.github.com/users/davidgilbertson/gists{/gist_id}
https://github.com/davidgilbertson
davidgilbertson
MDQ6VXNlcjQ0NDM0ODI=
https://api.github.com/users/davidgilbertson/orgs
https://api.github.com/users/davidgilbertson/received_events
https://api.github.com/users/davidgilbertson/repos
https://api.github.com/users/davidgilbertson/starred{/owner}{/repo}
https://api.github.com/users/davidgilbertson/subscriptions
User
https://api.github.com/users/davidgilbertson
public
### Describe the bug

I would like to share cache between two machines (a Windows host machine and a WSL instance).

I run most my code in WSL. I have just run out of space in the virtual drive. Rather than expand the drive size, I plan to move to cache to the host Windows machine, thereby sharing the downloads.

I'm hoping that I can just copy/paste the cache files, but I notice that a lot of the file names start with the path name, e.g. `_home_davidg_.cache_huggingface_datasets_conll2003_default-451...98.lock` where `home/davidg` is where the cache is in WSL. 

This seems to suggest that the cache is not portable/cannot be centralised or shared. Is this the case, or are the files that start with path names not integral to the caching mechanism? Because copying the cache files _seems_ to work, but I'm not filled with confidence that something isn't going to break.

A related issue, when trying to load a dataset that should come from cache (running in WSL, pointing to cache on the Windows host) it seemed to work fine, but it still uses a WSL directory for `.cache\huggingface\modules\datasets_modules`. I see nothing in the docs about this, or how to point it to a different place.

I have asked a related question on the forum: https://discuss.huggingface.co/t/is-datasets-cache-operating-system-agnostic/32656

### Steps to reproduce the bug

View the cache directory in WSL/Windows.

### Expected behavior

Cache can be shared between (virtual) machines and be transportable.

It would be nice to have a simple way to say "Dear Hugging Face packages, please put ALL your cache in `blah/de/blah`" and have all the Hugging Face packages respect that single location.

### Environment info

```
- `datasets` version: 2.9.0
- Platform: Linux-5.10.102.1-microsoft-standard-WSL2-x86_64-with-glibc2.31
- Python version: 3.10.8
- PyArrow version: 11.0.0
- Pandas version: 1.5.3
- ```