Fix fingerprint when moving cache dir
MEMBER
Windows, why are you doing this to me ?
2021-06-16T18:12:03Z
https://github.com/huggingface/datasets/pull/2509#issuecomment-862601213
https://api.github.com/repos/huggingface/datasets/issues/2509
MDEyOklzc3VlQ29tbWVudDg2MjYwMTIxMw==
https://api.github.com/repos/huggingface/datasets/issues/comments/862601213/reactions
2021-06-16T18:12:24Z
https://api.github.com/repos/huggingface/datasets/issues/comments/862601213
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
Thanks @lhoestq, I'm starting reviewing this PR.
2021-06-17T15:05:22Z
https://github.com/huggingface/datasets/pull/2509#issuecomment-863318529
https://api.github.com/repos/huggingface/datasets/issues/2509
MDEyOklzc3VlQ29tbWVudDg2MzMxODUyOQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/863318529/reactions
2021-06-17T15:05:22Z
https://api.github.com/repos/huggingface/datasets/issues/comments/863318529
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
MEMBER
Yea issues on windows are about long paths, not long filenames.
We can make sure the lock filenames are not too long, but not for the paths
2021-06-21T10:00:24Z
https://github.com/huggingface/datasets/pull/2509#issuecomment-864903542
https://api.github.com/repos/huggingface/datasets/issues/2509
MDEyOklzc3VlQ29tbWVudDg2NDkwMzU0Mg==
https://api.github.com/repos/huggingface/datasets/issues/comments/864903542/reactions
2021-06-21T10:00:24Z
https://api.github.com/repos/huggingface/datasets/issues/comments/864903542
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
Took your suggestions into account @albertvillanova :)
2021-06-21T10:30:55Z
https://github.com/huggingface/datasets/pull/2509#issuecomment-864923950
https://api.github.com/repos/huggingface/datasets/issues/2509
MDEyOklzc3VlQ29tbWVudDg2NDkyMzk1MA==
https://api.github.com/repos/huggingface/datasets/issues/comments/864923950/reactions
2021-06-21T10:30:55Z
https://api.github.com/repos/huggingface/datasets/issues/comments/864923950
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
The fingerprint of a dataset changes if the cache directory is moved.
I fixed that by setting the fingerprint to be the hash of:
- the relative cache dir (dataset_name/version/config_id)
- the requested split

Close #2496 

I had to fix an issue with the filelock filename that was too long (>255). It prevented the tests to run on my machine. I just added `hash_filename_if_too_long` in case this happens, to not get filenames longer than 255.
We usually have long filenames for filelocks because they are named after the path that is being locked. In case the path is a cache directory that has long directory names, then the filelock filename could en up being very long.