NonMatchingSplitsSizesError when loading blog_authorship_corpus
MEMBER
I just ran it on colab and got this
```
[{'expected': SplitInfo(name='train', num_bytes=610252351, num_examples=532812,
dataset_name='blog_authorship_corpus'), 'recorded': SplitInfo(name='train',
num_bytes=611607465, num_examples=533285, dataset_name='blog_authorship_corpus')},
{'expected': SplitInfo(name='validation', num_bytes=37500394, num_examples=31277,
dataset_name='blog_authorship_corpus'), 'recorded': SplitInfo(name='validation',
num_bytes=35652716, num_examples=30804, dataset_name='blog_authorship_corpus')}]
```
which is different from the `dataset_infos.json` and also different from yours.

It looks like the script for generating examples is not consistent
2020-05-29T13:07:45Z
https://github.com/huggingface/datasets/issues/215#issuecomment-635962609
https://api.github.com/repos/huggingface/datasets/issues/215
MDEyOklzc3VlQ29tbWVudDYzNTk2MjYwOQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/635962609/reactions
2020-05-29T13:08:07Z
https://api.github.com/repos/huggingface/datasets/issues/comments/635962609
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
The files provided by the authors are corrupted and the script seems to ignore the xml files that can't be decoded (it does `try:... except UnicodeDecodeError`). Maybe depending of the environment some files can be opened and some others don't but not sure why
2020-05-29T13:40:07Z
https://github.com/huggingface/datasets/issues/215#issuecomment-635978228
https://api.github.com/repos/huggingface/datasets/issues/215
MDEyOklzc3VlQ29tbWVudDYzNTk3ODIyOA==
https://api.github.com/repos/huggingface/datasets/issues/comments/635978228/reactions
2020-05-29T13:40:07Z
https://api.github.com/repos/huggingface/datasets/issues/comments/635978228
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
Feel free to do `ignore_verifications=True` for now... The verifications only include a check on the checksums of the downloaded files, and a check on the number of examples in each splits.
2020-05-29T13:42:17Z
https://github.com/huggingface/datasets/issues/215#issuecomment-635979227
https://api.github.com/repos/huggingface/datasets/issues/215
MDEyOklzc3VlQ29tbWVudDYzNTk3OTIyNw==
https://api.github.com/repos/huggingface/datasets/issues/comments/635979227/reactions
2020-12-30T17:26:11Z
https://api.github.com/repos/huggingface/datasets/issues/comments/635979227
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
I'm getting this same issue when loading the `imdb` corpus via `dataset = load_dataset("imdb")`. When I try `ignore_verifications=True`, no examples are read into the `train` portion of the dataset. 
2020-12-30T16:22:16Z
https://github.com/huggingface/datasets/issues/215#issuecomment-752679490
https://api.github.com/repos/huggingface/datasets/issues/215
MDEyOklzc3VlQ29tbWVudDc1MjY3OTQ5MA==
https://api.github.com/repos/huggingface/datasets/issues/comments/752679490/reactions
2020-12-30T16:22:16Z
https://api.github.com/repos/huggingface/datasets/issues/comments/752679490
https://avatars.githubusercontent.com/u/7334040?v=4
https://api.github.com/users/EmilyAlsentzer/events{/privacy}
https://api.github.com/users/EmilyAlsentzer/followers
https://api.github.com/users/EmilyAlsentzer/following{/other_user}
https://api.github.com/users/EmilyAlsentzer/gists{/gist_id}
https://github.com/EmilyAlsentzer
EmilyAlsentzer
MDQ6VXNlcjczMzQwNDA=
https://api.github.com/users/EmilyAlsentzer/orgs
https://api.github.com/users/EmilyAlsentzer/received_events
https://api.github.com/users/EmilyAlsentzer/repos
https://api.github.com/users/EmilyAlsentzer/starred{/owner}{/repo}
https://api.github.com/users/EmilyAlsentzer/subscriptions
User
https://api.github.com/users/EmilyAlsentzer
public
MEMBER
> I'm getting this same issue when loading the `imdb` corpus via `dataset = load_dataset("imdb")`. When I try `ignore_verifications=True`, no examples are read into the `train` portion of the dataset.

When the checksums don't match, it may mean that the file you downloaded is corrupted. In this case you can try to load the dataset again `load_dataset("imdb", download_mode="force_redownload")`

Also I just checked on my side and it worked fine:

```python
from datasets import load_dataset

dataset = load_dataset("imdb")
print(len(dataset["train"]))
# 25000
```

Let me know if redownloading fixes your issue @EmilyAlsentzer .
If not, feel free to open a separate issue.
2020-12-30T17:34:45Z
https://github.com/huggingface/datasets/issues/215#issuecomment-752700544
https://api.github.com/repos/huggingface/datasets/issues/215
MDEyOklzc3VlQ29tbWVudDc1MjcwMDU0NA==
https://api.github.com/repos/huggingface/datasets/issues/comments/752700544/reactions
2020-12-30T17:34:45Z
https://api.github.com/repos/huggingface/datasets/issues/comments/752700544
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
It doesn't seem to fix the problem. I'll open a separate issue. Thanks. 
2021-01-06T15:55:12Z
https://github.com/huggingface/datasets/issues/215#issuecomment-755383247
https://api.github.com/repos/huggingface/datasets/issues/215
MDEyOklzc3VlQ29tbWVudDc1NTM4MzI0Nw==
https://api.github.com/repos/huggingface/datasets/issues/comments/755383247/reactions
2021-01-06T15:55:12Z
https://api.github.com/repos/huggingface/datasets/issues/comments/755383247
https://avatars.githubusercontent.com/u/7334040?v=4
https://api.github.com/users/EmilyAlsentzer/events{/privacy}
https://api.github.com/users/EmilyAlsentzer/followers
https://api.github.com/users/EmilyAlsentzer/following{/other_user}
https://api.github.com/users/EmilyAlsentzer/gists{/gist_id}
https://github.com/EmilyAlsentzer
EmilyAlsentzer
MDQ6VXNlcjczMzQwNDA=
https://api.github.com/users/EmilyAlsentzer/orgs
https://api.github.com/users/EmilyAlsentzer/received_events
https://api.github.com/users/EmilyAlsentzer/repos
https://api.github.com/users/EmilyAlsentzer/starred{/owner}{/repo}
https://api.github.com/users/EmilyAlsentzer/subscriptions
User
https://api.github.com/users/EmilyAlsentzer
public
NONE
I wasn't aware of the "force_redownload" option and manually removed the '/home/me/.cache/huggingface/datasets/' dir, this worked for me (dataset 'cnn_dailymail')
2021-01-06T20:58:34Z
https://github.com/huggingface/datasets/issues/215#issuecomment-755685858
https://api.github.com/repos/huggingface/datasets/issues/215
MDEyOklzc3VlQ29tbWVudDc1NTY4NTg1OA==
https://api.github.com/repos/huggingface/datasets/issues/comments/755685858/reactions
2021-01-06T20:58:34Z
https://api.github.com/repos/huggingface/datasets/issues/comments/755685858
https://avatars.githubusercontent.com/u/11347638?v=4
https://api.github.com/users/putssander/events{/privacy}
https://api.github.com/users/putssander/followers
https://api.github.com/users/putssander/following{/other_user}
https://api.github.com/users/putssander/gists{/gist_id}
https://github.com/putssander
putssander
MDQ6VXNlcjExMzQ3NjM4
https://api.github.com/users/putssander/orgs
https://api.github.com/users/putssander/received_events
https://api.github.com/users/putssander/repos
https://api.github.com/users/putssander/starred{/owner}{/repo}
https://api.github.com/users/putssander/subscriptions
User
https://api.github.com/users/putssander
public
MEMBER
Yes I think this might not be documented well enough. Let’s add it to the doc @lhoestq @SBrandeis.
And everything on how to control the cache behavior better (removing, overriding, changing the path, etc)
2021-01-06T21:37:03Z
https://github.com/huggingface/datasets/issues/215#issuecomment-755729604
https://api.github.com/repos/huggingface/datasets/issues/215
MDEyOklzc3VlQ29tbWVudDc1NTcyOTYwNA==
https://api.github.com/repos/huggingface/datasets/issues/comments/755729604/reactions
2021-01-06T21:37:03Z
https://api.github.com/repos/huggingface/datasets/issues/comments/755729604
https://avatars.githubusercontent.com/u/7353373?v=4
https://api.github.com/users/thomwolf/events{/privacy}
https://api.github.com/users/thomwolf/followers
https://api.github.com/users/thomwolf/following{/other_user}
https://api.github.com/users/thomwolf/gists{/gist_id}
https://github.com/thomwolf
thomwolf
MDQ6VXNlcjczNTMzNzM=
https://api.github.com/users/thomwolf/orgs
https://api.github.com/users/thomwolf/received_events
https://api.github.com/users/thomwolf/repos
https://api.github.com/users/thomwolf/starred{/owner}{/repo}
https://api.github.com/users/thomwolf/subscriptions
User
https://api.github.com/users/thomwolf
public
MEMBER
Already fixed:
```python
In [1]: from datasets import load_dataset

In [2]: ds = load_dataset("blog_authorship_corpus")

In [3]: ds
Out[3]: 
DatasetDict({
    train: Dataset({
        features: ['text', 'date', 'gender', 'age', 'horoscope', 'job'],
        num_rows: 689793
    })
    validation: Dataset({
        features: ['text', 'date', 'gender', 'age', 'horoscope', 'job'],
        num_rows: 37919
    })
})

2022-02-10T13:05:45Z
https://github.com/huggingface/datasets/issues/215#issuecomment-1034900186
https://api.github.com/repos/huggingface/datasets/issues/215
IC_kwDODunzps49r1La
https://api.github.com/repos/huggingface/datasets/issues/comments/1034900186/reactions
2022-02-10T13:05:45Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1034900186
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
NONE
In my case, I had to remove the cache datasets directory completely as @putssander suggested, the download_mode='forced_redownload' was insufficient.

I had a private repository with data files that I loaded with a loading script. It was working fine until I pushed a new version of the data files and then the NonMatchingSplitsSizesError was raised.

2023-03-30T15:16:44Z
https://github.com/huggingface/datasets/issues/215#issuecomment-1490488022
https://api.github.com/repos/huggingface/datasets/issues/215
IC_kwDODunzps5Y1wrW
https://api.github.com/repos/huggingface/datasets/issues/comments/1490488022/reactions
2023-03-30T15:16:44Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1490488022
https://avatars.githubusercontent.com/u/45097509?v=4
https://api.github.com/users/JanRodriguez/events{/privacy}
https://api.github.com/users/JanRodriguez/followers
https://api.github.com/users/JanRodriguez/following{/other_user}
https://api.github.com/users/JanRodriguez/gists{/gist_id}
https://github.com/JanRodriguez
JanRodriguez
MDQ6VXNlcjQ1MDk3NTA5
https://api.github.com/users/JanRodriguez/orgs
https://api.github.com/users/JanRodriguez/received_events
https://api.github.com/users/JanRodriguez/repos
https://api.github.com/users/JanRodriguez/starred{/owner}{/repo}
https://api.github.com/users/JanRodriguez/subscriptions
User
https://api.github.com/users/JanRodriguez
public
NONE
I had the same issue. Adding` verification_mode="no_checks"` in laod_dataset() as param works for me 
2024-10-14T14:09:53Z
https://github.com/huggingface/datasets/issues/215#issuecomment-2411383605
https://api.github.com/repos/huggingface/datasets/issues/215
IC_kwDODunzps6Puss1
https://api.github.com/repos/huggingface/datasets/issues/comments/2411383605/reactions
2024-10-14T14:09:53Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2411383605
https://avatars.githubusercontent.com/u/95047673?v=4
https://api.github.com/users/Xiaomin-HUANG/events{/privacy}
https://api.github.com/users/Xiaomin-HUANG/followers
https://api.github.com/users/Xiaomin-HUANG/following{/other_user}
https://api.github.com/users/Xiaomin-HUANG/gists{/gist_id}
https://github.com/Xiaomin-HUANG
Xiaomin-HUANG
U_kgDOBapP-Q
https://api.github.com/users/Xiaomin-HUANG/orgs
https://api.github.com/users/Xiaomin-HUANG/received_events
https://api.github.com/users/Xiaomin-HUANG/repos
https://api.github.com/users/Xiaomin-HUANG/starred{/owner}{/repo}
https://api.github.com/users/Xiaomin-HUANG/subscriptions
User
https://api.github.com/users/Xiaomin-HUANG
public
NONE
i had the same issue, solved it by "
dataset = load_dataset(
    "cnn_dailymail",
    "3.0.0",
    download_mode="force_redownload",
    ignore_verifications=True
)
" and the dataset downloaded without a problem: DatasetDict({
    train: Dataset({
        features: ['article', 'highlights', 'id'],
        num_rows: 861339
    })
    validation: Dataset({
        features: ['article', 'highlights', 'id'],
        num_rows: 40104
    })
    test: Dataset({
        features: ['article', 'highlights', 'id'],
        num_rows: 34470
    })
})
2025-01-04T00:03:11Z
https://github.com/huggingface/datasets/issues/215#issuecomment-2569948500
https://api.github.com/repos/huggingface/datasets/issues/215
IC_kwDODunzps6ZLk1U
https://api.github.com/repos/huggingface/datasets/issues/comments/2569948500/reactions
2025-01-04T00:03:11Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2569948500
https://avatars.githubusercontent.com/u/124644691?v=4
https://api.github.com/users/goknurarican/events{/privacy}
https://api.github.com/users/goknurarican/followers
https://api.github.com/users/goknurarican/following{/other_user}
https://api.github.com/users/goknurarican/gists{/gist_id}
https://github.com/goknurarican
goknurarican
U_kgDOB23tUw
https://api.github.com/users/goknurarican/orgs
https://api.github.com/users/goknurarican/received_events
https://api.github.com/users/goknurarican/repos
https://api.github.com/users/goknurarican/starred{/owner}{/repo}
https://api.github.com/users/goknurarican/subscriptions
User
https://api.github.com/users/goknurarican
public
Getting this error when i run `nlp.load_dataset('blog_authorship_corpus')`. 

```
raise NonMatchingSplitsSizesError(str(bad_splits))
nlp.utils.info_utils.NonMatchingSplitsSizesError: [{'expected': SplitInfo(name='train', 
num_bytes=610252351, num_examples=532812, dataset_name='blog_authorship_corpus'), 
'recorded': SplitInfo(name='train', num_bytes=616473500, num_examples=536323, 
dataset_name='blog_authorship_corpus')}, {'expected': SplitInfo(name='validation', 
num_bytes=37500394, num_examples=31277, dataset_name='blog_authorship_corpus'), 
'recorded': SplitInfo(name='validation', num_bytes=30786661, num_examples=27766, 
dataset_name='blog_authorship_corpus')}]
```

Upon checking it seems like there is a disparity between the information in `datasets/blog_authorship_corpus/dataset_infos.json` and what was downloaded. Although I can get away with this by passing `ignore_verifications=True` in `load_dataset`, I'm thinking doing so might give problems later on.