Test dill
NONE
_The documentation is not available anymore as the PR was closed or merged._
2022-05-21T09:10:12Z
https://github.com/huggingface/datasets/pull/4385#issuecomment-1133585117
https://api.github.com/repos/huggingface/datasets/issues/4385
IC_kwDODunzps5DkSLd
https://api.github.com/repos/huggingface/datasets/issues/comments/1133585117/reactions
2022-05-25T08:30:13Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1133585117
https://avatars.githubusercontent.com/u/99929124?v=4
https://api.github.com/users/HuggingFaceDocBuilderDev/events{/privacy}
https://api.github.com/users/HuggingFaceDocBuilderDev/followers
https://api.github.com/users/HuggingFaceDocBuilderDev/following{/other_user}
https://api.github.com/users/HuggingFaceDocBuilderDev/gists{/gist_id}
https://github.com/HuggingFaceDocBuilderDev
HuggingFaceDocBuilderDev
U_kgDOBfTMJA
https://api.github.com/users/HuggingFaceDocBuilderDev/orgs
https://api.github.com/users/HuggingFaceDocBuilderDev/received_events
https://api.github.com/users/HuggingFaceDocBuilderDev/repos
https://api.github.com/users/HuggingFaceDocBuilderDev/starred{/owner}{/repo}
https://api.github.com/users/HuggingFaceDocBuilderDev/subscriptions
User
https://api.github.com/users/HuggingFaceDocBuilderDev
public
NONE
I should point out that the hash will be the same if computed twice with the same code on the same version of dill (after adding huggingface's code that removes line numbers and file names, and sorts globals.) My changes in dill 0.3.5 and ones that I will make in 0.3.6 will result in different pickles than the ones dill 0.3.4 was making. This should still be fine for caching.
2022-05-21T18:36:28Z
https://github.com/huggingface/datasets/pull/4385#issuecomment-1133749280
https://api.github.com/repos/huggingface/datasets/issues/4385
IC_kwDODunzps5Dk6Qg
https://api.github.com/repos/huggingface/datasets/issues/comments/1133749280/reactions
2022-05-21T18:36:28Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1133749280
https://avatars.githubusercontent.com/u/42706068?v=4
https://api.github.com/users/anivegesana/events{/privacy}
https://api.github.com/users/anivegesana/followers
https://api.github.com/users/anivegesana/following{/other_user}
https://api.github.com/users/anivegesana/gists{/gist_id}
https://github.com/anivegesana
anivegesana
MDQ6VXNlcjQyNzA2MDY4
https://api.github.com/users/anivegesana/orgs
https://api.github.com/users/anivegesana/received_events
https://api.github.com/users/anivegesana/repos
https://api.github.com/users/anivegesana/starred{/owner}{/repo}
https://api.github.com/users/anivegesana/subscriptions
User
https://api.github.com/users/anivegesana
public
MEMBER
Just some comments @lhoestq:

The best practice for testing is to have a `test_<filename>.py` for each `<filename>.py`. Therefore in order to have the filenames aligned, I would propose:
- either renaming `fingerprint.py` to `caching.py`
- or renaming `test_caching.py` to `test_fingerprint.py`

On the other hand, my idea when implementing this test was not to test all the functionalities of the `Hasher`, but just to have a regression test that fails if dill version is > 0.3.4 and the pin in our `setup.py` is not present. Just recall that we had no failing test in our CI when the issue with dill was found on `transformers`.

The objective of this PR is just to have a regression test for that case: I tested and I got `AttributeError: module 'dill._dill' has no attribute 'stack'`

For this regression test, I took into account this comment by @gugarosa: https://github.com/huggingface/datasets/issues/4379#issuecomment-1133131825

There is no equivalent test in `test_caching.py` because our CI did not fail before pinning dill.
2022-05-24T16:05:41Z
https://github.com/huggingface/datasets/pull/4385#issuecomment-1136117573
https://api.github.com/repos/huggingface/datasets/issues/4385
IC_kwDODunzps5Dt8dF
https://api.github.com/repos/huggingface/datasets/issues/comments/1136117573/reactions
2022-05-24T16:15:25Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1136117573
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
MEMBER
Ok I see, renaming it to `test_fingerprint.py` sounds like a good idea :)
2022-05-24T17:04:32Z
https://github.com/huggingface/datasets/pull/4385#issuecomment-1136193965
https://api.github.com/repos/huggingface/datasets/issues/4385
IC_kwDODunzps5DuPGt
https://api.github.com/repos/huggingface/datasets/issues/comments/1136193965/reactions
2022-05-24T17:04:32Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1136193965
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
Regression test for future releases of `dill`.

Related to #4379. 