Fix cache path to snakecase for `CachedDatasetModuleFactory` and `Cache`
CONTRIBUTOR
@lhoestq hi 😃, is there something else I need to do to check this change?
2024-03-28T15:49:50Z
https://github.com/huggingface/datasets/pull/6754#issuecomment-2025546537
https://api.github.com/repos/huggingface/datasets/issues/6754
IC_kwDODunzps54u2Mp
https://api.github.com/repos/huggingface/datasets/issues/comments/2025546537/reactions
2024-03-28T15:49:50Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2025546537
https://avatars.githubusercontent.com/u/26690193?v=4
https://api.github.com/users/izhx/events{/privacy}
https://api.github.com/users/izhx/followers
https://api.github.com/users/izhx/following{/other_user}
https://api.github.com/users/izhx/gists{/gist_id}
https://github.com/izhx
izhx
MDQ6VXNlcjI2NjkwMTkz
https://api.github.com/users/izhx/orgs
https://api.github.com/users/izhx/received_events
https://api.github.com/users/izhx/repos
https://api.github.com/users/izhx/starred{/owner}{/repo}
https://api.github.com/users/izhx/subscriptions
User
https://api.github.com/users/izhx
public
CONTRIBUTOR
I added two tests and passed them on my server.

```
pytest tests/packaged_modules/test_cache.py 
========================================================================== test session starts ==========================================================================
platform linux -- Python 3.11.5, pytest-8.1.1, pluggy-1.4.0
rootdir: /mnt/nas/datasets
configfile: pyproject.toml
plugins: xdist-3.5.0, datadir-1.5.0
collected 8 items                                                                                                                                                       

tests/packaged_modules/test_cache.py ........                                                                                                                     [100%]

========================================================================== 8 passed in 50.71s ===========================================================================

```

```
pytest tests/test_load.py
========================================================================== test session starts ==========================================================================
platform linux -- Python 3.11.5, pytest-8.1.1, pluggy-1.4.0
rootdir: /mnt/nas/datasets
configfile: pyproject.toml
plugins: xdist-3.5.0, datadir-1.5.0
collected 151 items                                                                                                                                                     

tests/test_load.py .............................................................................................................................................. [ 94%]
.........                                                                                                                                                         [100%]

...

============================================================= 151 passed, 29 warnings in 578.36s (0:09:38) ==============================================================
```

2024-03-29T02:52:39Z
https://github.com/huggingface/datasets/pull/6754#issuecomment-2026524189
https://api.github.com/repos/huggingface/datasets/issues/6754
IC_kwDODunzps54yk4d
https://api.github.com/repos/huggingface/datasets/issues/comments/2026524189/reactions
2024-03-29T02:52:39Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2026524189
https://avatars.githubusercontent.com/u/26690193?v=4
https://api.github.com/users/izhx/events{/privacy}
https://api.github.com/users/izhx/followers
https://api.github.com/users/izhx/following{/other_user}
https://api.github.com/users/izhx/gists{/gist_id}
https://github.com/izhx
izhx
MDQ6VXNlcjI2NjkwMTkz
https://api.github.com/users/izhx/orgs
https://api.github.com/users/izhx/received_events
https://api.github.com/users/izhx/repos
https://api.github.com/users/izhx/starred{/owner}{/repo}
https://api.github.com/users/izhx/subscriptions
User
https://api.github.com/users/izhx
public
NONE
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_6754). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
2024-03-29T11:45:59Z
https://github.com/huggingface/datasets/pull/6754#issuecomment-2027128394
https://api.github.com/repos/huggingface/datasets/issues/6754
IC_kwDODunzps5404ZK
https://api.github.com/repos/huggingface/datasets/issues/comments/2027128394/reactions
2024-03-29T11:45:59Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2027128394
https://avatars.githubusercontent.com/u/99929124?v=4
https://api.github.com/users/HuggingFaceDocBuilderDev/events{/privacy}
https://api.github.com/users/HuggingFaceDocBuilderDev/followers
https://api.github.com/users/HuggingFaceDocBuilderDev/following{/other_user}
https://api.github.com/users/HuggingFaceDocBuilderDev/gists{/gist_id}
https://github.com/HuggingFaceDocBuilderDev
HuggingFaceDocBuilderDev
U_kgDOBfTMJA
https://api.github.com/users/HuggingFaceDocBuilderDev/orgs
https://api.github.com/users/HuggingFaceDocBuilderDev/received_events
https://api.github.com/users/HuggingFaceDocBuilderDev/repos
https://api.github.com/users/HuggingFaceDocBuilderDev/starred{/owner}{/repo}
https://api.github.com/users/HuggingFaceDocBuilderDev/subscriptions
User
https://api.github.com/users/HuggingFaceDocBuilderDev
public
NONE
Hi @izhx! I have also faced this issue, happy to see it already addressed, looking forward for PR merge :)
2024-04-02T10:42:40Z
https://github.com/huggingface/datasets/pull/6754#issuecomment-2031665452
https://api.github.com/repos/huggingface/datasets/issues/6754
IC_kwDODunzps55GMEs
https://api.github.com/repos/huggingface/datasets/issues/comments/2031665452/reactions
2024-04-02T10:42:40Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2031665452
https://avatars.githubusercontent.com/u/6318579?v=4
https://api.github.com/users/minotru/events{/privacy}
https://api.github.com/users/minotru/followers
https://api.github.com/users/minotru/following{/other_user}
https://api.github.com/users/minotru/gists{/gist_id}
https://github.com/minotru
minotru
MDQ6VXNlcjYzMTg1Nzk=
https://api.github.com/users/minotru/orgs
https://api.github.com/users/minotru/received_events
https://api.github.com/users/minotru/repos
https://api.github.com/users/minotru/starred{/owner}{/repo}
https://api.github.com/users/minotru/subscriptions
User
https://api.github.com/users/minotru
public
CONTRIBUTOR
@lhoestq What do you think of these tests? 😀
2024-04-11T11:47:22Z
https://github.com/huggingface/datasets/pull/6754#issuecomment-2049519554
https://api.github.com/repos/huggingface/datasets/issues/6754
IC_kwDODunzps56KS_C
https://api.github.com/repos/huggingface/datasets/issues/comments/2049519554/reactions
2024-04-11T11:47:22Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2049519554
https://avatars.githubusercontent.com/u/26690193?v=4
https://api.github.com/users/izhx/events{/privacy}
https://api.github.com/users/izhx/followers
https://api.github.com/users/izhx/following{/other_user}
https://api.github.com/users/izhx/gists{/gist_id}
https://github.com/izhx
izhx
MDQ6VXNlcjI2NjkwMTkz
https://api.github.com/users/izhx/orgs
https://api.github.com/users/izhx/received_events
https://api.github.com/users/izhx/repos
https://api.github.com/users/izhx/starred{/owner}{/repo}
https://api.github.com/users/izhx/subscriptions
User
https://api.github.com/users/izhx
public
NONE
<details>
<summary>Show benchmarks</summary>

PyArrow==8.0.0

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.005060 / 0.011353 (-0.006293) | 0.003251 / 0.011008 (-0.007757) | 0.063538 / 0.038508 (0.025030) | 0.031178 / 0.023109 (0.008069) | 0.249971 / 0.275898 (-0.025927) | 0.284828 / 0.323480 (-0.038652) | 0.004183 / 0.007986 (-0.003802) | 0.002656 / 0.004328 (-0.001673) | 0.049585 / 0.004250 (0.045335) | 0.042656 / 0.037052 (0.005604) | 0.270962 / 0.258489 (0.012473) | 0.296091 / 0.293841 (0.002250) | 0.028065 / 0.128546 (-0.100482) | 0.010545 / 0.075646 (-0.065102) | 0.207323 / 0.419271 (-0.211948) | 0.035977 / 0.043533 (-0.007556) | 0.257315 / 0.255139 (0.002176) | 0.272238 / 0.283200 (-0.010962) | 0.017984 / 0.141683 (-0.123699) | 1.131314 / 1.452155 (-0.320840) | 1.180259 / 1.492716 (-0.312457) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.090977 / 0.018006 (0.072971) | 0.284021 / 0.000490 (0.283531) | 0.000264 / 0.000200 (0.000065) | 0.000044 / 0.000054 (-0.000011) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.017852 / 0.037411 (-0.019559) | 0.061288 / 0.014526 (0.046762) | 0.073844 / 0.176557 (-0.102713) | 0.121371 / 0.737135 (-0.615764) | 0.075036 / 0.296338 (-0.221303) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.287599 / 0.215209 (0.072390) | 2.821172 / 2.077655 (0.743517) | 1.459904 / 1.504120 (-0.044216) | 1.340224 / 1.541195 (-0.200970) | 1.357350 / 1.468490 (-0.111140) | 0.557344 / 4.584777 (-4.027433) | 2.412177 / 3.745712 (-1.333535) | 2.745126 / 5.269862 (-2.524735) | 1.754600 / 4.565676 (-2.811077) | 0.062487 / 0.424275 (-0.361788) | 0.005306 / 0.007607 (-0.002301) | 0.338856 / 0.226044 (0.112811) | 3.354953 / 2.268929 (1.086024) | 1.803208 / 55.444624 (-53.641417) | 1.553051 / 6.876477 (-5.323426) | 1.554790 / 2.142072 (-0.587282) | 0.651380 / 4.805227 (-4.153847) | 0.117777 / 6.500664 (-6.382887) | 0.041992 / 0.075469 (-0.033477) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.977588 / 1.841788 (-0.864200) | 11.363058 / 8.074308 (3.288750) | 9.791770 / 10.191392 (-0.399622) | 0.130708 / 0.680424 (-0.549716) | 0.013798 / 0.534201 (-0.520403) | 0.288313 / 0.579283 (-0.290970) | 0.268170 / 0.434364 (-0.166194) | 0.324815 / 0.540337 (-0.215522) | 0.419260 / 1.386936 (-0.967676) |

</details>
PyArrow==latest

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.005187 / 0.011353 (-0.006166) | 0.003348 / 0.011008 (-0.007660) | 0.050309 / 0.038508 (0.011801) | 0.031334 / 0.023109 (0.008225) | 0.279542 / 0.275898 (0.003644) | 0.299608 / 0.323480 (-0.023872) | 0.004202 / 0.007986 (-0.003784) | 0.002735 / 0.004328 (-0.001593) | 0.050321 / 0.004250 (0.046070) | 0.039793 / 0.037052 (0.002740) | 0.289972 / 0.258489 (0.031483) | 0.313887 / 0.293841 (0.020046) | 0.028797 / 0.128546 (-0.099750) | 0.010166 / 0.075646 (-0.065480) | 0.059228 / 0.419271 (-0.360044) | 0.032667 / 0.043533 (-0.010866) | 0.278409 / 0.255139 (0.023270) | 0.292208 / 0.283200 (0.009008) | 0.017577 / 0.141683 (-0.124106) | 1.175046 / 1.452155 (-0.277109) | 1.200766 / 1.492716 (-0.291950) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.092236 / 0.018006 (0.074230) | 0.298860 / 0.000490 (0.298370) | 0.000211 / 0.000200 (0.000011) | 0.000043 / 0.000054 (-0.000011) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.021475 / 0.037411 (-0.015936) | 0.074414 / 0.014526 (0.059888) | 0.087746 / 0.176557 (-0.088811) | 0.124757 / 0.737135 (-0.612378) | 0.088513 / 0.296338 (-0.207826) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.296583 / 0.215209 (0.081374) | 2.894978 / 2.077655 (0.817323) | 1.590806 / 1.504120 (0.086686) | 1.463251 / 1.541195 (-0.077944) | 1.478751 / 1.468490 (0.010261) | 0.571724 / 4.584777 (-4.013053) | 2.454356 / 3.745712 (-1.291356) | 2.789275 / 5.269862 (-2.480586) | 1.753866 / 4.565676 (-2.811811) | 0.064787 / 0.424275 (-0.359488) | 0.005321 / 0.007607 (-0.002287) | 0.348454 / 0.226044 (0.122410) | 3.453052 / 2.268929 (1.184124) | 1.972237 / 55.444624 (-53.472388) | 1.677822 / 6.876477 (-5.198655) | 1.674750 / 2.142072 (-0.467322) | 0.649353 / 4.805227 (-4.155874) | 0.117135 / 6.500664 (-6.383529) | 0.040018 / 0.075469 (-0.035451) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.029812 / 1.841788 (-0.811976) | 11.945063 / 8.074308 (3.870755) | 10.238380 / 10.191392 (0.046988) | 0.146225 / 0.680424 (-0.534199) | 0.015262 / 0.534201 (-0.518939) | 0.286632 / 0.579283 (-0.292651) | 0.272952 / 0.434364 (-0.161412) | 0.323098 / 0.540337 (-0.217239) | 0.423549 / 1.386936 (-0.963387) |

</details>
</details>

![](https://cml.dev/watermark.png#91b07b90915d7f7313d44ca3ff67673b9ad26bf4 "CML watermark")

2024-04-15T15:45:43Z
https://github.com/huggingface/datasets/pull/6754#issuecomment-2057175329
https://api.github.com/repos/huggingface/datasets/issues/6754
IC_kwDODunzps56ngEh
Iv1.05c79e9ad1f6bdfa
2018-07-30T09:30:17Z
Automate your workflow from idea to production
branch_protection_rule check_run check_suite create delete deployment deployment_status discussion discussion_comment fork gollum issues issue_comment label merge_group milestone page_build project project_card project_column public pull_request pull_request_review pull_request_review_comment push registry_package release repository repository_dispatch status watch workflow_dispatch workflow_run
https://help.github.com/en/actions
https://github.com/apps/github-actions
GitHub Actions
MDM6QXBwMTUzNjg=
https://avatars.githubusercontent.com/u/9919?v=4
https://api.github.com/users/github/events{/privacy}
https://api.github.com/users/github/followers
https://api.github.com/users/github/following{/other_user}
https://api.github.com/users/github/gists{/gist_id}
https://github.com/github
github
MDEyOk9yZ2FuaXphdGlvbjk5MTk=
https://api.github.com/users/github/orgs
https://api.github.com/users/github/received_events
https://api.github.com/users/github/repos
https://api.github.com/users/github/starred{/owner}{/repo}
https://api.github.com/users/github/subscriptions
Organization
https://api.github.com/users/github
public
write
read
write
write
write
write
write
write
write
read
read
write
write
write
write
write
write
write
read
github-actions
2025-03-07T16:35:00Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2057175329/reactions
2024-04-15T15:45:43Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2057175329
https://avatars.githubusercontent.com/in/15368?v=4
https://api.github.com/users/github-actions%5Bbot%5D/events{/privacy}
https://api.github.com/users/github-actions%5Bbot%5D/followers
https://api.github.com/users/github-actions%5Bbot%5D/following{/other_user}
https://api.github.com/users/github-actions%5Bbot%5D/gists{/gist_id}
https://github.com/apps/github-actions
github-actions[bot]
MDM6Qm90NDE4OTgyODI=
https://api.github.com/users/github-actions%5Bbot%5D/orgs
https://api.github.com/users/github-actions%5Bbot%5D/received_events
https://api.github.com/users/github-actions%5Bbot%5D/repos
https://api.github.com/users/github-actions%5Bbot%5D/starred{/owner}{/repo}
https://api.github.com/users/github-actions%5Bbot%5D/subscriptions
Bot
https://api.github.com/users/github-actions%5Bbot%5D
public
Fix https://github.com/huggingface/datasets/issues/6750#issuecomment-2016678729

I didn't find a guideline on how to run the tests, so i just run the following steps to make sure that this bug is fixed.
1. `python test.py`,
2. then `HF_DATASETS_OFFLINE=1 python test.py`

The `test.py` is
```
  import datasets

  datasets.utils.logging.set_verbosity_info()

  ds = datasets.load_dataset('izhx/STS17-debug')
  print(ds)

  ds = datasets.load_dataset('C-MTEB/AFQMC', revision='b44c3b011063adb25877c13823db83bb193913c4')
  print(ds)
```


