Datasets method `.map` not hashing
CONTRIBUTOR
Fix posted: https://github.com/huggingface/datasets/issues/4506#issuecomment-1157417219
2022-06-17T11:39:05Z
https://github.com/huggingface/datasets/issues/4521#issuecomment-1158786199
https://api.github.com/repos/huggingface/datasets/issues/4521
IC_kwDODunzps5FEayX
https://api.github.com/repos/huggingface/datasets/issues/comments/1158786199/reactions
2022-06-17T11:39:05Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1158786199
https://avatars.githubusercontent.com/u/93869735?v=4
https://api.github.com/users/sanchit-gandhi/events{/privacy}
https://api.github.com/users/sanchit-gandhi/followers
https://api.github.com/users/sanchit-gandhi/following{/other_user}
https://api.github.com/users/sanchit-gandhi/gists{/gist_id}
https://github.com/sanchit-gandhi
sanchit-gandhi
U_kgDOBZhWpw
https://api.github.com/users/sanchit-gandhi/orgs
https://api.github.com/users/sanchit-gandhi/received_events
https://api.github.com/users/sanchit-gandhi/repos
https://api.github.com/users/sanchit-gandhi/starred{/owner}{/repo}
https://api.github.com/users/sanchit-gandhi/subscriptions
User
https://api.github.com/users/sanchit-gandhi
public
NONE
Didn't realize it's a bug when I asked the question yesterday! Free free to post an answer if you are sure the cause has been addressed.

https://stackoverflow.com/questions/72664827/can-pickle-dill-foo-but-not-lambda-x-foox
2022-06-18T22:10:30Z
https://github.com/huggingface/datasets/issues/4521#issuecomment-1159573360
https://api.github.com/repos/huggingface/datasets/issues/4521
IC_kwDODunzps5FHa9w
https://api.github.com/repos/huggingface/datasets/issues/comments/1159573360/reactions
2022-06-18T22:10:30Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1159573360
https://avatars.githubusercontent.com/u/13443062?v=4
https://api.github.com/users/nalzok/events{/privacy}
https://api.github.com/users/nalzok/followers
https://api.github.com/users/nalzok/following{/other_user}
https://api.github.com/users/nalzok/gists{/gist_id}
https://github.com/nalzok
nalzok
MDQ6VXNlcjEzNDQzMDYy
https://api.github.com/users/nalzok/orgs
https://api.github.com/users/nalzok/received_events
https://api.github.com/users/nalzok/repos
https://api.github.com/users/nalzok/starred{/owner}{/repo}
https://api.github.com/users/nalzok/subscriptions
User
https://api.github.com/users/nalzok
public
NONE
Thank @nalzok . That works for me:

`pip install "dill<0.3.5"`
2022-08-04T12:07:40Z
https://github.com/huggingface/datasets/issues/4521#issuecomment-1205166653
https://api.github.com/repos/huggingface/datasets/issues/4521
IC_kwDODunzps5H1WI9
https://api.github.com/repos/huggingface/datasets/issues/comments/1205166653/reactions
2022-08-04T12:08:16Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1205166653
https://avatars.githubusercontent.com/u/100429654?v=4
https://api.github.com/users/diahpitaloka/events{/privacy}
https://api.github.com/users/diahpitaloka/followers
https://api.github.com/users/diahpitaloka/following{/other_user}
https://api.github.com/users/diahpitaloka/gists{/gist_id}
https://github.com/diahpitaloka
diahpitaloka
U_kgDOBfxvVg
https://api.github.com/users/diahpitaloka/orgs
https://api.github.com/users/diahpitaloka/received_events
https://api.github.com/users/diahpitaloka/repos
https://api.github.com/users/diahpitaloka/starred{/owner}{/repo}
https://api.github.com/users/diahpitaloka/subscriptions
User
https://api.github.com/users/diahpitaloka
public
## Describe the bug
Datasets method `.map` not hashing, even with an empty no-op function

## Steps to reproduce the bug
```python
from datasets import load_dataset

# download 9MB dummy dataset
ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean")

def prepare_dataset(batch):
     return(batch)

ds = ds.map(
    prepare_dataset,
    num_proc=1,
    desc="preprocess train dataset",
)
```

## Expected results
Hashed and cached dataset preprocessing

## Actual results
Does not hash properly:
```
Parameter 'function'=<function prepare_dataset at 0x7fccb68e9280> of the transform datasets.arrow_dataset.Dataset._map_single couldn't be hashed properly, a random hash was used instead. Make sure your transforms and parameters are serializable with pickle or dill for the dataset fingerprinting and caching to work. If you reuse this transform, the caching mechanism will consider it to be different from the previous calls and recompute everything. This warning is only showed once. Subsequent hashing failures won't be showed.
```

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 2.3.3.dev0
- Platform: Linux-5.11.0-1028-gcp-x86_64-with-glibc2.31
- Python version: 3.9.12
- PyArrow version: 8.0.0
- Pandas version: 1.4.2

cc @lhoestq 
