My new dataset PEC
CONTRIBUTOR
How to resolve these failed checks?
2020-11-29T12:52:56Z
https://github.com/huggingface/datasets/pull/913#issuecomment-735389270
https://api.github.com/repos/huggingface/datasets/issues/913
MDEyOklzc3VlQ29tbWVudDczNTM4OTI3MA==
https://api.github.com/repos/huggingface/datasets/issues/comments/735389270/reactions
2020-11-29T12:52:56Z
https://api.github.com/repos/huggingface/datasets/issues/comments/735389270
https://avatars.githubusercontent.com/u/11826803?v=4
https://api.github.com/users/zhongpeixiang/events{/privacy}
https://api.github.com/users/zhongpeixiang/followers
https://api.github.com/users/zhongpeixiang/following{/other_user}
https://api.github.com/users/zhongpeixiang/gists{/gist_id}
https://github.com/zhongpeixiang
zhongpeixiang
MDQ6VXNlcjExODI2ODAz
https://api.github.com/users/zhongpeixiang/orgs
https://api.github.com/users/zhongpeixiang/received_events
https://api.github.com/users/zhongpeixiang/repos
https://api.github.com/users/zhongpeixiang/starred{/owner}{/repo}
https://api.github.com/users/zhongpeixiang/subscriptions
User
https://api.github.com/users/zhongpeixiang
public
MEMBER
Thanks for adding this one :) 

To fix the check_code_quality, please run `make style` with the latest version of black, isort, flake8
To fix the test_no_encoding_on_file_open, make sure to specify the encoding each time you call `open()` on a text file.
For example : `encoding="utf-8"`
To fix the test_load_dataset_pec , you must add the dummy_data.zip file. It is used to test the dataset script and make sure it runs fine. To add it, please refer to the steps in https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-add-a-dataset


2020-11-29T23:09:28Z
https://github.com/huggingface/datasets/pull/913#issuecomment-735472622
https://api.github.com/repos/huggingface/datasets/issues/913
MDEyOklzc3VlQ29tbWVudDczNTQ3MjYyMg==
https://api.github.com/repos/huggingface/datasets/issues/comments/735472622/reactions
2020-11-29T23:09:28Z
https://api.github.com/repos/huggingface/datasets/issues/comments/735472622
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
Could you also add a dataset card ? you can find a template here : https://github.com/huggingface/datasets/blob/master/templates/README.md

That would be awesome
2020-11-29T23:10:09Z
https://github.com/huggingface/datasets/pull/913#issuecomment-735472743
https://api.github.com/repos/huggingface/datasets/issues/913
MDEyOklzc3VlQ29tbWVudDczNTQ3Mjc0Mw==
https://api.github.com/repos/huggingface/datasets/issues/comments/735472743/reactions
2020-11-29T23:10:09Z
https://api.github.com/repos/huggingface/datasets/issues/comments/735472743
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
> Thanks for adding this one :)
> 
> To fix the check_code_quality, please run `make style` with the latest version of black, isort, flake8
> To fix the test_no_encoding_on_file_open, make sure to specify the encoding each time you call `open()` on a text file.
> For example : `encoding="utf-8"`
> To fix the test_load_dataset_pec , you must add the dummy_data.zip file. It is used to test the dataset script and make sure it runs fine. To add it, please refer to the steps in https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-add-a-dataset

Thank you for the detailed suggestion.

I have added dummy_data but it still failed the DistributedDatasetTest check. My dataset has a central file (containing a python dict) that needs to be accessed by each data example. Is it because the central file cannot be distributed (which would lead to a partial dictionary)?

Specifically, the central file contains a dictionary of speakers with their attributes. Each data example is also associated with a speaker. As of now, I keep the central file and data files separately. If I remove the central file by appending the speaker attributes to each data example, then there would be lots of redundancy because there are lots of duplicate speakers in the data files.
2020-11-30T04:36:01Z
https://github.com/huggingface/datasets/pull/913#issuecomment-735539134
https://api.github.com/repos/huggingface/datasets/issues/913
MDEyOklzc3VlQ29tbWVudDczNTUzOTEzNA==
https://api.github.com/repos/huggingface/datasets/issues/comments/735539134/reactions
2020-11-30T04:42:07Z
https://api.github.com/repos/huggingface/datasets/issues/comments/735539134
https://avatars.githubusercontent.com/u/11826803?v=4
https://api.github.com/users/zhongpeixiang/events{/privacy}
https://api.github.com/users/zhongpeixiang/followers
https://api.github.com/users/zhongpeixiang/following{/other_user}
https://api.github.com/users/zhongpeixiang/gists{/gist_id}
https://github.com/zhongpeixiang
zhongpeixiang
MDQ6VXNlcjExODI2ODAz
https://api.github.com/users/zhongpeixiang/orgs
https://api.github.com/users/zhongpeixiang/received_events
https://api.github.com/users/zhongpeixiang/repos
https://api.github.com/users/zhongpeixiang/starred{/owner}{/repo}
https://api.github.com/users/zhongpeixiang/subscriptions
User
https://api.github.com/users/zhongpeixiang
public
MEMBER
The `DistributedDatasetTest` fail and the changes of this PR are not related, there was just a bug in the CI. You can ignore it
2020-11-30T13:23:19Z
https://github.com/huggingface/datasets/pull/913#issuecomment-735782663
https://api.github.com/repos/huggingface/datasets/issues/913
MDEyOklzc3VlQ29tbWVudDczNTc4MjY2Mw==
https://api.github.com/repos/huggingface/datasets/issues/comments/735782663/reactions
2020-11-30T13:23:19Z
https://api.github.com/repos/huggingface/datasets/issues/comments/735782663
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
> Really cool thanks !
> 
> Could you make the dummy files smaller ? For example by reducing the size of persona.txt ?
> I also left a comment about the files concatenation. It would be cool to replace that with simple iterations through the different files.
> 
> Then once this is done, you can add a dataset card using the template guide here : https://github.com/huggingface/datasets/blob/master/templates/README_guide.md
> If some fields can't be filled, just leave `[N/A]`

Small change: if you don't have the information for a field, please leave `[More Information Needed]` rather than `[N/A]`

The full information can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md#manually-tag-the-dataset-and-write-the-dataset-card)
2020-11-30T15:42:01Z
https://github.com/huggingface/datasets/pull/913#issuecomment-735864949
https://api.github.com/repos/huggingface/datasets/issues/913
MDEyOklzc3VlQ29tbWVudDczNTg2NDk0OQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/735864949/reactions
2020-11-30T15:42:01Z
https://api.github.com/repos/huggingface/datasets/issues/comments/735864949
https://avatars.githubusercontent.com/u/10469459?v=4
https://api.github.com/users/yjernite/events{/privacy}
https://api.github.com/users/yjernite/followers
https://api.github.com/users/yjernite/following{/other_user}
https://api.github.com/users/yjernite/gists{/gist_id}
https://github.com/yjernite
yjernite
MDQ6VXNlcjEwNDY5NDU5
https://api.github.com/users/yjernite/orgs
https://api.github.com/users/yjernite/received_events
https://api.github.com/users/yjernite/repos
https://api.github.com/users/yjernite/starred{/owner}{/repo}
https://api.github.com/users/yjernite/subscriptions
User
https://api.github.com/users/yjernite
public
A new dataset PEC published in EMNLP 2020.