Support pyarrow LargeListType
NONE
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_6835). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
2024-04-24T11:38:58Z
https://github.com/huggingface/datasets/pull/6835#issuecomment-2074743085
https://api.github.com/repos/huggingface/datasets/issues/6835
IC_kwDODunzps57qhEt
https://api.github.com/repos/huggingface/datasets/issues/comments/2074743085/reactions
2024-04-24T11:38:58Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2074743085
https://avatars.githubusercontent.com/u/99929124?v=4
https://api.github.com/users/HuggingFaceDocBuilderDev/events{/privacy}
https://api.github.com/users/HuggingFaceDocBuilderDev/followers
https://api.github.com/users/HuggingFaceDocBuilderDev/following{/other_user}
https://api.github.com/users/HuggingFaceDocBuilderDev/gists{/gist_id}
https://github.com/HuggingFaceDocBuilderDev
HuggingFaceDocBuilderDev
U_kgDOBfTMJA
https://api.github.com/users/HuggingFaceDocBuilderDev/orgs
https://api.github.com/users/HuggingFaceDocBuilderDev/received_events
https://api.github.com/users/HuggingFaceDocBuilderDev/repos
https://api.github.com/users/HuggingFaceDocBuilderDev/starred{/owner}{/repo}
https://api.github.com/users/HuggingFaceDocBuilderDev/subscriptions
User
https://api.github.com/users/HuggingFaceDocBuilderDev
public
CONTRIBUTOR
Fixed the conversion from `pyarrow` to `python` `Sequence` features. 

There is still an issue that if `features` are passed the `Sequence` always forces conversion to `ListArray`.
This probably causes issues if the `LargeListArray` is actually needed.

There doesn't seem to be a great solution since this list is created solely on the `schema` for `Sequence`.
One solution would be to always use `LargeListArray` instead.

2024-04-24T13:45:05Z
https://github.com/huggingface/datasets/pull/6835#issuecomment-2074989285
https://api.github.com/repos/huggingface/datasets/issues/6835
IC_kwDODunzps57rdLl
https://api.github.com/repos/huggingface/datasets/issues/comments/2074989285/reactions
2024-04-24T13:54:03Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2074989285
https://avatars.githubusercontent.com/u/37351874?v=4
https://api.github.com/users/Modexus/events{/privacy}
https://api.github.com/users/Modexus/followers
https://api.github.com/users/Modexus/following{/other_user}
https://api.github.com/users/Modexus/gists{/gist_id}
https://github.com/Modexus
Modexus
MDQ6VXNlcjM3MzUxODc0
https://api.github.com/users/Modexus/orgs
https://api.github.com/users/Modexus/received_events
https://api.github.com/users/Modexus/repos
https://api.github.com/users/Modexus/starred{/owner}{/repo}
https://api.github.com/users/Modexus/subscriptions
User
https://api.github.com/users/Modexus
public
MEMBER
I am retaking this PR because we would like to have this feature implemented.
2024-07-01T08:46:10Z
https://github.com/huggingface/datasets/pull/6835#issuecomment-2199582777
https://api.github.com/repos/huggingface/datasets/issues/6835
IC_kwDODunzps6DGvg5
https://api.github.com/repos/huggingface/datasets/issues/comments/2199582777/reactions
2024-07-01T08:46:10Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2199582777
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
Fixes #6834 