medical-dialog error
MEMBER
Hi @smeyerhot, thanks for reporting.

You are right: there is an issue with the dataset metadata. I'm fixing it.

In the meantime, you can circumvent the issue by passing `ignore_verifications=True`:
```python
raw_datasets = load_dataset("medical_dialog", "en", split="train", download_mode="force_redownload", data_dir="./Medical-Dialogue-Dataset-English", ignore_verifications=True)
```
2021-10-08T07:49:13Z
https://github.com/huggingface/datasets/issues/2969#issuecomment-938423278
https://api.github.com/repos/huggingface/datasets/issues/2969
IC_kwDODunzps437zPu
https://api.github.com/repos/huggingface/datasets/issues/comments/938423278/reactions
2021-10-08T07:49:13Z
https://api.github.com/repos/huggingface/datasets/issues/comments/938423278
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
NONE
Hi @albertvillanova -- I see you made changes to the wikitext dataset a few hours ago.
I'm getting an error similar to the one here : https://github.com/huggingface/datasets/issues/215
After your changes. 

I deleted the original dataset cache after encountering that error and then rerun with the following command :

`traindata = load_dataset('wikitext', 'wikitext-2-raw-v1', split='train', ignore_verifications=True, download_mode='force_redownload')
    testdata = load_dataset('wikitext', 'wikitext-2-raw-v1', split='test', ignore_verifications=True)`

But this is currently hanging.  Any ideas ? Thanks
2024-01-05T07:31:45Z
https://github.com/huggingface/datasets/issues/2969#issuecomment-1878247042
https://api.github.com/repos/huggingface/datasets/issues/2969
IC_kwDODunzps5v88aC
https://api.github.com/repos/huggingface/datasets/issues/comments/1878247042/reactions
2024-01-05T07:32:02Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1878247042
https://avatars.githubusercontent.com/u/38739988?v=4
https://api.github.com/users/ldery/events{/privacy}
https://api.github.com/users/ldery/followers
https://api.github.com/users/ldery/following{/other_user}
https://api.github.com/users/ldery/gists{/gist_id}
https://github.com/ldery
ldery
MDQ6VXNlcjM4NzM5OTg4
https://api.github.com/users/ldery/orgs
https://api.github.com/users/ldery/received_events
https://api.github.com/users/ldery/repos
https://api.github.com/users/ldery/starred{/owner}{/repo}
https://api.github.com/users/ldery/subscriptions
User
https://api.github.com/users/ldery
public
MEMBER
@ldery, you can find the answer in the corresponding Discussion on the Hub: https://huggingface.co/datasets/wikitext/discussions/9

You need to update the `datasets` library:
```shell
pip install -U datasets
```
2024-01-08T09:55:10Z
https://github.com/huggingface/datasets/issues/2969#issuecomment-1880682940
https://api.github.com/repos/huggingface/datasets/issues/2969
IC_kwDODunzps5wGPG8
https://api.github.com/repos/huggingface/datasets/issues/comments/1880682940/reactions
2024-01-08T09:55:10Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1880682940
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
## Describe the bug
A clear and concise description of what the bug is.
When I attempt to download the huggingface datatset medical_dialog it errors out midway through
## Steps to reproduce the bug
```python
raw_datasets = load_dataset("medical_dialog", "en", split="train", download_mode="force_redownload", data_dir="./Medical-Dialogue-Dataset-English")
```

## Expected results
A clear and concise description of the expected results.
No error
## Actual results

```
3 frames
/usr/local/lib/python3.7/dist-packages/datasets/utils/info_utils.py in verify_splits(expected_splits, recorded_splits)
     72     ]
     73     if len(bad_splits) > 0:
---> 74         raise NonMatchingSplitsSizesError(str(bad_splits))
     75     logger.info("All the splits matched successfully.")
     76 

NonMatchingSplitsSizesError: [{'expected': SplitInfo(name='train', num_bytes=0, num_examples=0, dataset_name='medical_dialog'), 'recorded': SplitInfo(name='train', num_bytes=295097913, num_examples=229674, dataset_name='medical_dialog')}]
```
Specify the actual results or traceback.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.21.1
- Platform: colab
- Python version: colab 3.7
- PyArrow version: N/A
