Import error on Google Colab
COLLABORATOR
Hi! This can happen if an incompatible `pyarrow` version (`pyarrow<12.0.0`) has been imported before the `datasets` installation and the Colab session hasn't been restarted afterward. To avoid the error, go to "Runtime -> Restart session" after `!pip install -U datasets` and before `import datasets`, or insert the `import os; os.kill(os.getpid(), 9)` cell between `!pip install -U datasets` and `import datasets` to do the same programmatically.
2024-02-13T16:59:48Z
https://github.com/huggingface/datasets/issues/6661#issuecomment-1942010668
https://api.github.com/repos/huggingface/datasets/issues/6661
IC_kwDODunzps5zwLss
https://api.github.com/repos/huggingface/datasets/issues/comments/1942010668/reactions
2024-02-13T16:59:48Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1942010668
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
MEMBER
One possible cause might be the one pointed out by @mariosasko above, and you get the following warning on Colab:
```
WARNING: The following packages were previously imported in this runtime:
  [pyarrow]
You must restart the runtime in order to use newly installed versions.
```

On the other hand, if the old version of `pyarrow` is not previously imported (before the installation of `datasets`), the reported issue here is not reproducible: `datasets` can be installed, imported and used on Colab.
2024-02-14T08:04:47Z
https://github.com/huggingface/datasets/issues/6661#issuecomment-1943253514
https://api.github.com/repos/huggingface/datasets/issues/6661
IC_kwDODunzps5z07IK
https://api.github.com/repos/huggingface/datasets/issues/comments/1943253514/reactions
2024-02-14T08:04:47Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1943253514
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
MEMBER
Duplicate of:
- #5923
2024-02-16T14:43:43Z
https://github.com/huggingface/datasets/issues/6661#issuecomment-1948512177
https://api.github.com/repos/huggingface/datasets/issues/6661
IC_kwDODunzps50I--x
https://api.github.com/repos/huggingface/datasets/issues/comments/1948512177/reactions
2024-02-16T14:43:43Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1948512177
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
COLLABORATOR
Google Colab now pre-installs PyArrow 14.0.2, making this issue unlikely to happen. So, I'm unpinning it.
2024-02-25T16:34:41Z
https://github.com/huggingface/datasets/issues/6661#issuecomment-1962993250
https://api.github.com/repos/huggingface/datasets/issues/6661
IC_kwDODunzps51AOZi
https://api.github.com/repos/huggingface/datasets/issues/comments/1962993250/reactions
2024-02-25T16:37:54Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1962993250
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
### Describe the bug

Cannot be imported on Google Colab, the import throws the following error:
ValueError: pyarrow.lib.IpcWriteOptions size changed, may indicate binary incompatibility. Expected 88 from C header, got 72 from PyObject

### Steps to reproduce the bug

1. `! pip install -U datasets`
2. `import datasets`

### Expected behavior

Should be possible to use the library

### Environment info

- `datasets` version: 2.17.0
- Platform: Linux-6.1.58+-x86_64-with-glibc2.35
- Python version: 3.10.12
- `huggingface_hub` version: 0.20.3
- PyArrow version: 15.0.0
- Pandas version: 1.5.3
- `fsspec` version: 2023.6.0