Hind_Encorp all done
CONTRIBUTOR
ALL TEST PASSED locally  @yjernite 
2020-12-15T01:43:02Z
https://github.com/huggingface/datasets/pull/1575#issuecomment-744985256
https://api.github.com/repos/huggingface/datasets/issues/1575
MDEyOklzc3VlQ29tbWVudDc0NDk4NTI1Ng==
https://api.github.com/repos/huggingface/datasets/issues/comments/744985256/reactions
2020-12-15T01:49:36Z
https://api.github.com/repos/huggingface/datasets/issues/comments/744985256
https://avatars.githubusercontent.com/u/56379013?v=4
https://api.github.com/users/rahul-art/events{/privacy}
https://api.github.com/users/rahul-art/followers
https://api.github.com/users/rahul-art/following{/other_user}
https://api.github.com/users/rahul-art/gists{/gist_id}
https://github.com/rahul-art
rahul-art
MDQ6VXNlcjU2Mzc5MDEz
https://api.github.com/users/rahul-art/orgs
https://api.github.com/users/rahul-art/received_events
https://api.github.com/users/rahul-art/repos
https://api.github.com/users/rahul-art/starred{/owner}{/repo}
https://api.github.com/users/rahul-art/subscriptions
User
https://api.github.com/users/rahul-art
public
CONTRIBUTOR
@rahul-art  kindly run the following from the datasets folder 

```
make style 
flake8 datasets

```

2020-12-15T07:39:11Z
https://github.com/huggingface/datasets/pull/1575#issuecomment-745112991
https://api.github.com/repos/huggingface/datasets/issues/1575
MDEyOklzc3VlQ29tbWVudDc0NTExMjk5MQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/745112991/reactions
2020-12-15T07:39:11Z
https://api.github.com/repos/huggingface/datasets/issues/comments/745112991
https://avatars.githubusercontent.com/u/9033954?v=4
https://api.github.com/users/skyprince999/events{/privacy}
https://api.github.com/users/skyprince999/followers
https://api.github.com/users/skyprince999/following{/other_user}
https://api.github.com/users/skyprince999/gists{/gist_id}
https://github.com/skyprince999
skyprince999
MDQ6VXNlcjkwMzM5NTQ=
https://api.github.com/users/skyprince999/orgs
https://api.github.com/users/skyprince999/received_events
https://api.github.com/users/skyprince999/repos
https://api.github.com/users/skyprince999/starred{/owner}{/repo}
https://api.github.com/users/skyprince999/subscriptions
User
https://api.github.com/users/skyprince999
public
CONTRIBUTOR
@skyprince999 I did that before it says all done 

2020-12-15T09:42:23Z
https://github.com/huggingface/datasets/pull/1575#issuecomment-745172781
https://api.github.com/repos/huggingface/datasets/issues/1575
MDEyOklzc3VlQ29tbWVudDc0NTE3Mjc4MQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/745172781/reactions
2020-12-15T09:42:23Z
https://api.github.com/repos/huggingface/datasets/issues/comments/745172781
https://avatars.githubusercontent.com/u/56379013?v=4
https://api.github.com/users/rahul-art/events{/privacy}
https://api.github.com/users/rahul-art/followers
https://api.github.com/users/rahul-art/following{/other_user}
https://api.github.com/users/rahul-art/gists{/gist_id}
https://github.com/rahul-art
rahul-art
MDQ6VXNlcjU2Mzc5MDEz
https://api.github.com/users/rahul-art/orgs
https://api.github.com/users/rahul-art/received_events
https://api.github.com/users/rahul-art/repos
https://api.github.com/users/rahul-art/starred{/owner}{/repo}
https://api.github.com/users/rahul-art/subscriptions
User
https://api.github.com/users/rahul-art
public
CONTRIBUTOR
I did that again it gives the same output all done and then  I synchronized my changes with this branch 
2020-12-15T09:50:03Z
https://github.com/huggingface/datasets/pull/1575#issuecomment-745176944
https://api.github.com/repos/huggingface/datasets/issues/1575
MDEyOklzc3VlQ29tbWVudDc0NTE3Njk0NA==
https://api.github.com/repos/huggingface/datasets/issues/comments/745176944/reactions
2020-12-15T09:50:03Z
https://api.github.com/repos/huggingface/datasets/issues/comments/745176944
https://avatars.githubusercontent.com/u/56379013?v=4
https://api.github.com/users/rahul-art/events{/privacy}
https://api.github.com/users/rahul-art/followers
https://api.github.com/users/rahul-art/following{/other_user}
https://api.github.com/users/rahul-art/gists{/gist_id}
https://github.com/rahul-art
rahul-art
MDQ6VXNlcjU2Mzc5MDEz
https://api.github.com/users/rahul-art/orgs
https://api.github.com/users/rahul-art/received_events
https://api.github.com/users/rahul-art/repos
https://api.github.com/users/rahul-art/starred{/owner}{/repo}
https://api.github.com/users/rahul-art/subscriptions
User
https://api.github.com/users/rahul-art
public
CONTRIBUTOR
@lhoestq i did all the changes you suggested but at the time of load_dataset it is giving me error
`**`datasets.utils.info_utils.NonMatchingSplitsSizesError: [{'expected': SplitInfo(name='train', num_bytes=76591256, num_examples=1, dataset_name='hind_encorp'), 'recorded': SplitInfo(name='train', num_bytes=78945714, num_examples=273885, dataset_name='hind_encorp')}]`**`
2020-12-15T23:54:08Z
https://github.com/huggingface/datasets/pull/1575#issuecomment-745640471
https://api.github.com/repos/huggingface/datasets/issues/1575
MDEyOklzc3VlQ29tbWVudDc0NTY0MDQ3MQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/745640471/reactions
2020-12-15T23:54:08Z
https://api.github.com/repos/huggingface/datasets/issues/comments/745640471
https://avatars.githubusercontent.com/u/56379013?v=4
https://api.github.com/users/rahul-art/events{/privacy}
https://api.github.com/users/rahul-art/followers
https://api.github.com/users/rahul-art/following{/other_user}
https://api.github.com/users/rahul-art/gists{/gist_id}
https://github.com/rahul-art
rahul-art
MDQ6VXNlcjU2Mzc5MDEz
https://api.github.com/users/rahul-art/orgs
https://api.github.com/users/rahul-art/received_events
https://api.github.com/users/rahul-art/repos
https://api.github.com/users/rahul-art/starred{/owner}{/repo}
https://api.github.com/users/rahul-art/subscriptions
User
https://api.github.com/users/rahul-art
public
CONTRIBUTOR



I cloned the branch and it seems to work fine at my end. try to clear the cache - 

```
rm -rf  /home/ubuntu/.cache/huggingface/datasets/
rm -rf  /home/ubuntu/.cache/huggingface/modules/datasets_modules//datasets/
```
But the dataset has only one record. Is that correct? 
![image](https://user-images.githubusercontent.com/9033954/102331376-c7929b00-3fb0-11eb-8a6c-81b2cf47bc2a.png)

2020-12-16T09:40:20Z
https://github.com/huggingface/datasets/pull/1575#issuecomment-745995046
https://api.github.com/repos/huggingface/datasets/issues/1575
MDEyOklzc3VlQ29tbWVudDc0NTk5NTA0Ng==
https://api.github.com/repos/huggingface/datasets/issues/comments/745995046/reactions
2020-12-16T09:40:20Z
https://api.github.com/repos/huggingface/datasets/issues/comments/745995046
https://avatars.githubusercontent.com/u/9033954?v=4
https://api.github.com/users/skyprince999/events{/privacy}
https://api.github.com/users/skyprince999/followers
https://api.github.com/users/skyprince999/following{/other_user}
https://api.github.com/users/skyprince999/gists{/gist_id}
https://github.com/skyprince999
skyprince999
MDQ6VXNlcjkwMzM5NTQ=
https://api.github.com/users/skyprince999/orgs
https://api.github.com/users/skyprince999/received_events
https://api.github.com/users/skyprince999/repos
https://api.github.com/users/skyprince999/starred{/owner}{/repo}
https://api.github.com/users/skyprince999/subscriptions
User
https://api.github.com/users/skyprince999
public
MEMBER
> @lhoestq i did all the changes you suggested but at the time of load_dataset it is giving me error
> `**`datasets.utils.info_utils.NonMatchingSplitsSizesError: [{'expected': SplitInfo(name='train', num_bytes=76591256, num_examples=1, dataset_name='hind_encorp'), 'recorded': SplitInfo(name='train', num_bytes=78945714, num_examples=273885, dataset_name='hind_encorp')}]`**`

You can ignore this error by adding `ignore_verifications=True` to `load_dataset`.

This error is raised because you're loading a dataset that you've already loaded once in the past. Therefore the library does some verifications to make sure it's generated the same way. 

However since you've done changes in the dataset script you should ignore these verifications.

You can regenerate the dataset_infos.json with
```
datasets-cli test ./datasets/hindi_encorp --save_infos --all_configs --ignore_verifications
```


> I cloned the branch and it seems to work fine at my end. try to clear the cache -
> 
> ```
> rm -rf  /home/ubuntu/.cache/huggingface/datasets/
> rm -rf  /home/ubuntu/.cache/huggingface/modules/datasets_modules//datasets/
> ```
> 
> But the dataset has only one record. Is that correct?
> ![image](https://user-images.githubusercontent.com/9033954/102331376-c7929b00-3fb0-11eb-8a6c-81b2cf47bc2a.png)

Yes the current parsing is wrong, I've already given @rahul-art some suggestions and it looks like it works way better now (num_examples=273885).

Thanks for fixing the parsing @rahul-art !
Feel free to commit and push your changes once it's ready :) 
2020-12-16T09:48:50Z
https://github.com/huggingface/datasets/pull/1575#issuecomment-746004356
https://api.github.com/repos/huggingface/datasets/issues/1575
MDEyOklzc3VlQ29tbWVudDc0NjAwNDM1Ng==
https://api.github.com/repos/huggingface/datasets/issues/comments/746004356/reactions
2020-12-16T09:48:50Z
https://api.github.com/repos/huggingface/datasets/issues/comments/746004356
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
i ran the command you provided datasets-cli test ./datasets/hindi_encorp --save_infos --all_configs --ignore_verifications 
but now its giving this error @lhoestq 

FileNotFoundError: Couldn't find file locally at ./datasets/hindi_encorp/hindi_encorp.py, or remotely at https://raw.githubusercontent.com/huggingface/datasets/master/datasets/./datasets/hindi_encorp/hindi_encorp.py or https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/./datasets/hindi_encorp/hindi_encorp.py.
If the dataset was added recently, you may need to to pass script_version="master" to find the loading script on the master branch.

2020-12-16T10:56:20Z
https://github.com/huggingface/datasets/pull/1575#issuecomment-746096991
https://api.github.com/repos/huggingface/datasets/issues/1575
MDEyOklzc3VlQ29tbWVudDc0NjA5Njk5MQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/746096991/reactions
2020-12-16T10:56:45Z
https://api.github.com/repos/huggingface/datasets/issues/comments/746096991
https://avatars.githubusercontent.com/u/56379013?v=4
https://api.github.com/users/rahul-art/events{/privacy}
https://api.github.com/users/rahul-art/followers
https://api.github.com/users/rahul-art/following{/other_user}
https://api.github.com/users/rahul-art/gists{/gist_id}
https://github.com/rahul-art
rahul-art
MDQ6VXNlcjU2Mzc5MDEz
https://api.github.com/users/rahul-art/orgs
https://api.github.com/users/rahul-art/received_events
https://api.github.com/users/rahul-art/repos
https://api.github.com/users/rahul-art/starred{/owner}{/repo}
https://api.github.com/users/rahul-art/subscriptions
User
https://api.github.com/users/rahul-art
public
MEMBER
whoops I meant `hind_encorp` instead of `hindi_encorp` sorry
2020-12-16T10:57:23Z
https://github.com/huggingface/datasets/pull/1575#issuecomment-746098547
https://api.github.com/repos/huggingface/datasets/issues/1575
MDEyOklzc3VlQ29tbWVudDc0NjA5ODU0Nw==
https://api.github.com/repos/huggingface/datasets/issues/comments/746098547/reactions
2020-12-16T10:57:23Z
https://api.github.com/repos/huggingface/datasets/issues/comments/746098547
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
@lhoestq  all changes have done successfully in this PR #1584
2020-12-16T12:39:36Z
https://github.com/huggingface/datasets/pull/1575#issuecomment-746229150
https://api.github.com/repos/huggingface/datasets/issues/1575
MDEyOklzc3VlQ29tbWVudDc0NjIyOTE1MA==
https://api.github.com/repos/huggingface/datasets/issues/comments/746229150/reactions
2020-12-16T12:40:09Z
https://api.github.com/repos/huggingface/datasets/issues/comments/746229150
https://avatars.githubusercontent.com/u/56379013?v=4
https://api.github.com/users/rahul-art/events{/privacy}
https://api.github.com/users/rahul-art/followers
https://api.github.com/users/rahul-art/following{/other_user}
https://api.github.com/users/rahul-art/gists{/gist_id}
https://github.com/rahul-art
rahul-art
MDQ6VXNlcjU2Mzc5MDEz
https://api.github.com/users/rahul-art/orgs
https://api.github.com/users/rahul-art/received_events
https://api.github.com/users/rahul-art/repos
https://api.github.com/users/rahul-art/starred{/owner}{/repo}
https://api.github.com/users/rahul-art/subscriptions
User
https://api.github.com/users/rahul-art
public
MEMBER
Ok thanks ! closing this one in favor of #1584 
2020-12-16T15:15:17Z
https://github.com/huggingface/datasets/pull/1575#issuecomment-746453808
https://api.github.com/repos/huggingface/datasets/issues/1575
MDEyOklzc3VlQ29tbWVudDc0NjQ1MzgwOA==
https://api.github.com/repos/huggingface/datasets/issues/comments/746453808/reactions
2020-12-16T15:15:17Z
https://api.github.com/repos/huggingface/datasets/issues/comments/746453808
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public