Requirements should specify pyarrow<1
NONE
Hello @mathcass I would want to work on this issue. May I do the same? 
2020-10-04T15:21:54Z
https://github.com/huggingface/datasets/issues/707#issuecomment-703271074
https://api.github.com/repos/huggingface/datasets/issues/707
MDEyOklzc3VlQ29tbWVudDcwMzI3MTA3NA==
https://api.github.com/repos/huggingface/datasets/issues/comments/703271074/reactions
2020-10-04T15:21:54Z
https://api.github.com/repos/huggingface/datasets/issues/comments/703271074
https://avatars.githubusercontent.com/u/17764594?v=4
https://api.github.com/users/punitaojha/events{/privacy}
https://api.github.com/users/punitaojha/followers
https://api.github.com/users/punitaojha/following{/other_user}
https://api.github.com/users/punitaojha/gists{/gist_id}
https://github.com/punitaojha
punitaojha
MDQ6VXNlcjE3NzY0NTk0
https://api.github.com/users/punitaojha/orgs
https://api.github.com/users/punitaojha/received_events
https://api.github.com/users/punitaojha/repos
https://api.github.com/users/punitaojha/starred{/owner}{/repo}
https://api.github.com/users/punitaojha/subscriptions
User
https://api.github.com/users/punitaojha
public
NONE
@punitaojha, certainly. Feel free to work on this. Let me know if you need any help or clarity.
2020-10-04T17:51:01Z
https://github.com/huggingface/datasets/issues/707#issuecomment-703291059
https://api.github.com/repos/huggingface/datasets/issues/707
MDEyOklzc3VlQ29tbWVudDcwMzI5MTA1OQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/703291059/reactions
2020-10-04T17:51:01Z
https://api.github.com/repos/huggingface/datasets/issues/comments/703291059
https://avatars.githubusercontent.com/u/918541?v=4
https://api.github.com/users/mathcass/events{/privacy}
https://api.github.com/users/mathcass/followers
https://api.github.com/users/mathcass/following{/other_user}
https://api.github.com/users/mathcass/gists{/gist_id}
https://github.com/mathcass
mathcass
MDQ6VXNlcjkxODU0MQ==
https://api.github.com/users/mathcass/orgs
https://api.github.com/users/mathcass/received_events
https://api.github.com/users/mathcass/repos
https://api.github.com/users/mathcass/starred{/owner}{/repo}
https://api.github.com/users/mathcass/subscriptions
User
https://api.github.com/users/mathcass
public
NONE
Hello @mathcass 
1. I did fork the repository and clone the same on my local system. 

2. Then learnt about how we can publish our package on pypi.org. Also, found some instructions on same in setup.py documentation.

3. Then I Perplexity document link that you shared above. I created a colab link from there keep both tensorflow and pytorch means a mixed option and tried to run it in colab but I encountered no errors at a point where you mentioned. Can you help me to figure out the issue. 

4.Here is the link of the colab file with my saved responses. 
https://colab.research.google.com/drive/1hfYz8Ira39FnREbxgwa_goZWpOojp2NH?usp=sharing
2020-10-04T18:52:50Z
https://github.com/huggingface/datasets/issues/707#issuecomment-703299344
https://api.github.com/repos/huggingface/datasets/issues/707
MDEyOklzc3VlQ29tbWVudDcwMzI5OTM0NA==
https://api.github.com/repos/huggingface/datasets/issues/comments/703299344/reactions
2020-10-04T18:52:50Z
https://api.github.com/repos/huggingface/datasets/issues/comments/703299344
https://avatars.githubusercontent.com/u/17764594?v=4
https://api.github.com/users/punitaojha/events{/privacy}
https://api.github.com/users/punitaojha/followers
https://api.github.com/users/punitaojha/following{/other_user}
https://api.github.com/users/punitaojha/gists{/gist_id}
https://github.com/punitaojha
punitaojha
MDQ6VXNlcjE3NzY0NTk0
https://api.github.com/users/punitaojha/orgs
https://api.github.com/users/punitaojha/received_events
https://api.github.com/users/punitaojha/repos
https://api.github.com/users/punitaojha/starred{/owner}{/repo}
https://api.github.com/users/punitaojha/subscriptions
User
https://api.github.com/users/punitaojha
public
NONE
Also, please share some links which made you conclude that pyarrow < 1 would help. 
2020-10-04T18:54:02Z
https://github.com/huggingface/datasets/issues/707#issuecomment-703299482
https://api.github.com/repos/huggingface/datasets/issues/707
MDEyOklzc3VlQ29tbWVudDcwMzI5OTQ4Mg==
https://api.github.com/repos/huggingface/datasets/issues/comments/703299482/reactions
2020-10-04T18:54:02Z
https://api.github.com/repos/huggingface/datasets/issues/comments/703299482
https://avatars.githubusercontent.com/u/17764594?v=4
https://api.github.com/users/punitaojha/events{/privacy}
https://api.github.com/users/punitaojha/followers
https://api.github.com/users/punitaojha/following{/other_user}
https://api.github.com/users/punitaojha/gists{/gist_id}
https://github.com/punitaojha
punitaojha
MDQ6VXNlcjE3NzY0NTk0
https://api.github.com/users/punitaojha/orgs
https://api.github.com/users/punitaojha/received_events
https://api.github.com/users/punitaojha/repos
https://api.github.com/users/punitaojha/starred{/owner}{/repo}
https://api.github.com/users/punitaojha/subscriptions
User
https://api.github.com/users/punitaojha
public
NONE
Access granted for the colab link. 
2020-10-04T20:37:19Z
https://github.com/huggingface/datasets/issues/707#issuecomment-703312613
https://api.github.com/repos/huggingface/datasets/issues/707
MDEyOklzc3VlQ29tbWVudDcwMzMxMjYxMw==
https://api.github.com/repos/huggingface/datasets/issues/comments/703312613/reactions
2020-10-04T20:37:19Z
https://api.github.com/repos/huggingface/datasets/issues/comments/703312613
https://avatars.githubusercontent.com/u/17764594?v=4
https://api.github.com/users/punitaojha/events{/privacy}
https://api.github.com/users/punitaojha/followers
https://api.github.com/users/punitaojha/following{/other_user}
https://api.github.com/users/punitaojha/gists{/gist_id}
https://github.com/punitaojha
punitaojha
MDQ6VXNlcjE3NzY0NTk0
https://api.github.com/users/punitaojha/orgs
https://api.github.com/users/punitaojha/received_events
https://api.github.com/users/punitaojha/repos
https://api.github.com/users/punitaojha/starred{/owner}{/repo}
https://api.github.com/users/punitaojha/subscriptions
User
https://api.github.com/users/punitaojha
public
NONE
Thanks for looking at this @punitaojha and thanks for sharing the notebook. 

I just tried to reproduce this on my own (based on the environment where I had this issue) and I can't reproduce it somehow. If I run into this again, I'll include some steps to reproduce it. I'll close this as invalid. 

Thanks again. 
2020-10-04T20:50:28Z
https://github.com/huggingface/datasets/issues/707#issuecomment-703314217
https://api.github.com/repos/huggingface/datasets/issues/707
MDEyOklzc3VlQ29tbWVudDcwMzMxNDIxNw==
https://api.github.com/repos/huggingface/datasets/issues/comments/703314217/reactions
2020-10-04T20:50:28Z
https://api.github.com/repos/huggingface/datasets/issues/comments/703314217
https://avatars.githubusercontent.com/u/918541?v=4
https://api.github.com/users/mathcass/events{/privacy}
https://api.github.com/users/mathcass/followers
https://api.github.com/users/mathcass/following{/other_user}
https://api.github.com/users/mathcass/gists{/gist_id}
https://github.com/mathcass
mathcass
MDQ6VXNlcjkxODU0MQ==
https://api.github.com/users/mathcass/orgs
https://api.github.com/users/mathcass/received_events
https://api.github.com/users/mathcass/repos
https://api.github.com/users/mathcass/starred{/owner}{/repo}
https://api.github.com/users/mathcass/subscriptions
User
https://api.github.com/users/mathcass
public
NONE
I am sorry for hijacking this closed issue, but I believe I was able to reproduce this very issue. Strangely enough, it also turned out that running `pip install "pyarrow<1" --upgrade` did indeed fix the issue (PyArrow was installed in version `0.14.1` in my case).

Please see the Colab below:

https://colab.research.google.com/drive/15QQS3xWjlKW2aK0J74eEcRFuhXUddUST

Thanks!
2020-12-04T08:22:39Z
https://github.com/huggingface/datasets/issues/707#issuecomment-738639348
https://api.github.com/repos/huggingface/datasets/issues/707
MDEyOklzc3VlQ29tbWVudDczODYzOTM0OA==
https://api.github.com/repos/huggingface/datasets/issues/comments/738639348/reactions
2020-12-04T08:22:39Z
https://api.github.com/repos/huggingface/datasets/issues/comments/738639348
https://avatars.githubusercontent.com/u/461491?v=4
https://api.github.com/users/mrshu/events{/privacy}
https://api.github.com/users/mrshu/followers
https://api.github.com/users/mrshu/following{/other_user}
https://api.github.com/users/mrshu/gists{/gist_id}
https://github.com/mrshu
mrshu
MDQ6VXNlcjQ2MTQ5MQ==
https://api.github.com/users/mrshu/orgs
https://api.github.com/users/mrshu/received_events
https://api.github.com/users/mrshu/repos
https://api.github.com/users/mrshu/starred{/owner}{/repo}
https://api.github.com/users/mrshu/subscriptions
User
https://api.github.com/users/mrshu
public
I was looking at the docs on [Perplexity](https://huggingface.co/transformers/perplexity.html) via GPT2. When you load datasets and try to load Wikitext, you get the error,

```
module 'pyarrow' has no attribute 'PyExtensionType'
```
I traced it back to datasets having installed PyArrow 1.0.1 but there's not pinning in the setup file. 

https://github.com/huggingface/datasets/blob/e86a2a8f869b91654e782c9133d810bb82783200/setup.py#L68

Downgrading by installing `pip install "pyarrow<1"` resolved the issue.