add Indonesian newspapers (id_newspapers_2018)
MEMBER
Looks like there's a `Path` issue on windows. Could you try switching to
`glob.glob(os.path.join(article_dir, "*.json"))`
2020-12-10T22:27:52Z
https://github.com/huggingface/datasets/pull/1468#issuecomment-742839851
https://api.github.com/repos/huggingface/datasets/issues/1468
MDEyOklzc3VlQ29tbWVudDc0MjgzOTg1MQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/742839851/reactions
2020-12-10T22:27:52Z
https://api.github.com/repos/huggingface/datasets/issues/comments/742839851
https://avatars.githubusercontent.com/u/10469459?v=4
https://api.github.com/users/yjernite/events{/privacy}
https://api.github.com/users/yjernite/followers
https://api.github.com/users/yjernite/following{/other_user}
https://api.github.com/users/yjernite/gists{/gist_id}
https://github.com/yjernite
yjernite
MDQ6VXNlcjEwNDY5NDU5
https://api.github.com/users/yjernite/orgs
https://api.github.com/users/yjernite/received_events
https://api.github.com/users/yjernite/repos
https://api.github.com/users/yjernite/starred{/owner}{/repo}
https://api.github.com/users/yjernite/subscriptions
User
https://api.github.com/users/yjernite
public
CONTRIBUTOR
> Looks like there's a `Path` issue on windows. Could you try switching to
> `glob.glob(os.path.join(article_dir, "*.json"))`

Thanks, I replaced it with glob. Let's see if it solves the issue. Anyway, the main directory has a space, could it make the issue on windows? the test on linux don't have this problem.
2020-12-11T08:17:51Z
https://github.com/huggingface/datasets/pull/1468#issuecomment-743046350
https://api.github.com/repos/huggingface/datasets/issues/1468
MDEyOklzc3VlQ29tbWVudDc0MzA0NjM1MA==
https://api.github.com/repos/huggingface/datasets/issues/comments/743046350/reactions
2020-12-11T08:17:51Z
https://api.github.com/repos/huggingface/datasets/issues/comments/743046350
https://avatars.githubusercontent.com/u/7669893?v=4
https://api.github.com/users/cahya-wirawan/events{/privacy}
https://api.github.com/users/cahya-wirawan/followers
https://api.github.com/users/cahya-wirawan/following{/other_user}
https://api.github.com/users/cahya-wirawan/gists{/gist_id}
https://github.com/cahya-wirawan
cahya-wirawan
MDQ6VXNlcjc2Njk4OTM=
https://api.github.com/users/cahya-wirawan/orgs
https://api.github.com/users/cahya-wirawan/received_events
https://api.github.com/users/cahya-wirawan/repos
https://api.github.com/users/cahya-wirawan/starred{/owner}{/repo}
https://api.github.com/users/cahya-wirawan/subscriptions
User
https://api.github.com/users/cahya-wirawan
public
CONTRIBUTOR
It seems glob doesn't help also. Btw, one of the failing test tried to connect aws which failed:
```
C:\tools\miniconda3\lib\site-packages\urllib3\connection.py:160: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

address = ('s3.amazonaws.com', 443), timeout = 10, source_address = None
socket_options = [(6, 1, 1)]

```
Why did it try to connect to aws? I don't use it.
2020-12-11T08:36:39Z
https://github.com/huggingface/datasets/pull/1468#issuecomment-743055179
https://api.github.com/repos/huggingface/datasets/issues/1468
MDEyOklzc3VlQ29tbWVudDc0MzA1NTE3OQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/743055179/reactions
2020-12-11T08:44:35Z
https://api.github.com/repos/huggingface/datasets/issues/comments/743055179
https://avatars.githubusercontent.com/u/7669893?v=4
https://api.github.com/users/cahya-wirawan/events{/privacy}
https://api.github.com/users/cahya-wirawan/followers
https://api.github.com/users/cahya-wirawan/following{/other_user}
https://api.github.com/users/cahya-wirawan/gists{/gist_id}
https://github.com/cahya-wirawan
cahya-wirawan
MDQ6VXNlcjc2Njk4OTM=
https://api.github.com/users/cahya-wirawan/orgs
https://api.github.com/users/cahya-wirawan/received_events
https://api.github.com/users/cahya-wirawan/repos
https://api.github.com/users/cahya-wirawan/starred{/owner}{/repo}
https://api.github.com/users/cahya-wirawan/subscriptions
User
https://api.github.com/users/cahya-wirawan
public
CONTRIBUTOR
It seems that the circleci make a test for whole datasets repository, that means if only one of the dataset in the official repository has a download issue, this will also affect the test of a new dataset like mine, isn't it?
I changed the url to my newspaper dataset which contains only few simple json files and simple directory structure. But it still failed. And it failed not only on windows test. This is one of the error message:
```
-- Docs: https://docs.pytest.org/en/stable/warnings.html
=========================== short test summary info ============================
FAILED tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_ajgt_twitter_ar
FAILED tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_chr_en
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_load_dataset_ajgt_twitter_ar
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_load_dataset_chr_en
===== 4 failed, 2667 passed, 2052 skipped, 4 warnings in 432.05s (0:07:12) =====

Exited with code exit status 1
CircleCI received exit code 1
```
The test failed on twitter dataset even my dataset has nothing to do with twitter? 
2020-12-11T11:15:21Z
https://github.com/huggingface/datasets/pull/1468#issuecomment-743134398
https://api.github.com/repos/huggingface/datasets/issues/1468
MDEyOklzc3VlQ29tbWVudDc0MzEzNDM5OA==
https://api.github.com/repos/huggingface/datasets/issues/comments/743134398/reactions
2020-12-11T11:15:21Z
https://api.github.com/repos/huggingface/datasets/issues/comments/743134398
https://avatars.githubusercontent.com/u/7669893?v=4
https://api.github.com/users/cahya-wirawan/events{/privacy}
https://api.github.com/users/cahya-wirawan/followers
https://api.github.com/users/cahya-wirawan/following{/other_user}
https://api.github.com/users/cahya-wirawan/gists{/gist_id}
https://github.com/cahya-wirawan
cahya-wirawan
MDQ6VXNlcjc2Njk4OTM=
https://api.github.com/users/cahya-wirawan/orgs
https://api.github.com/users/cahya-wirawan/received_events
https://api.github.com/users/cahya-wirawan/repos
https://api.github.com/users/cahya-wirawan/starred{/owner}{/repo}
https://api.github.com/users/cahya-wirawan/subscriptions
User
https://api.github.com/users/cahya-wirawan
public
MEMBER
merging since the CI is fixed on master
2020-12-11T17:04:02Z
https://github.com/huggingface/datasets/pull/1468#issuecomment-743312689
https://api.github.com/repos/huggingface/datasets/issues/1468
MDEyOklzc3VlQ29tbWVudDc0MzMxMjY4OQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/743312689/reactions
2020-12-11T17:04:02Z
https://api.github.com/repos/huggingface/datasets/issues/comments/743312689
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Hi, thanks for merging the dataset. I create a new PR (#1499) since I need to update the link to the dataset. 
2020-12-12T08:50:51Z
https://github.com/huggingface/datasets/pull/1468#issuecomment-743725835
https://api.github.com/repos/huggingface/datasets/issues/1468
MDEyOklzc3VlQ29tbWVudDc0MzcyNTgzNQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/743725835/reactions
2020-12-12T08:50:51Z
https://api.github.com/repos/huggingface/datasets/issues/comments/743725835
https://avatars.githubusercontent.com/u/7669893?v=4
https://api.github.com/users/cahya-wirawan/events{/privacy}
https://api.github.com/users/cahya-wirawan/followers
https://api.github.com/users/cahya-wirawan/following{/other_user}
https://api.github.com/users/cahya-wirawan/gists{/gist_id}
https://github.com/cahya-wirawan
cahya-wirawan
MDQ6VXNlcjc2Njk4OTM=
https://api.github.com/users/cahya-wirawan/orgs
https://api.github.com/users/cahya-wirawan/received_events
https://api.github.com/users/cahya-wirawan/repos
https://api.github.com/users/cahya-wirawan/starred{/owner}{/repo}
https://api.github.com/users/cahya-wirawan/subscriptions
User
https://api.github.com/users/cahya-wirawan
public
The dataset contains around 500K articles (136M of words) from 7 Indonesian newspapers. The size of uncompressed 500K json files (newspapers-json.tgz) is around 2.2GB.