`Sequence(Features(...))` causes PyArrow cast error in `load_dataset` despite correct schema.
NONE
Hi @lhoestq 

Could you help confirm whether this qualifies as a bug?

It looks like the issue stems from how `Sequence(Features(...))` is interpreted as a plain struct during schema inference, which leads to a mismatch when casting with PyArrow (especially with nested structs inside lists). From the description, this seems like an inconsistency with expected behavior.

If confirmed, I’d be happy to take a shot at investigating and potentially submitting a fix.

Also looping in @AHS-uni  — could you kindly share a minimal JSONL example that reproduces this?

Thanks!
2025-05-30T09:02:10Z
https://github.com/huggingface/datasets/issues/7590#issuecomment-2921704887
https://api.github.com/repos/huggingface/datasets/issues/7590
IC_kwDODunzps6uJa23
https://api.github.com/repos/huggingface/datasets/issues/comments/2921704887/reactions
2025-05-30T09:02:10Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2921704887
https://avatars.githubusercontent.com/u/180720690?v=4
https://api.github.com/users/Flink-ddd/events{/privacy}
https://api.github.com/users/Flink-ddd/followers
https://api.github.com/users/Flink-ddd/following{/other_user}
https://api.github.com/users/Flink-ddd/gists{/gist_id}
https://github.com/Flink-ddd
Flink-ddd
U_kgDOCsWUMg
https://api.github.com/users/Flink-ddd/orgs
https://api.github.com/users/Flink-ddd/received_events
https://api.github.com/users/Flink-ddd/repos
https://api.github.com/users/Flink-ddd/starred{/owner}{/repo}
https://api.github.com/users/Flink-ddd/subscriptions
User
https://api.github.com/users/Flink-ddd
public
NONE
Hello @Flink-ddd 

I updated the minimal example and included both JSON and JSONL minimal examples in the Colab notebook. 

Here is the minimal  JSON file for convenience (can't upload JSONL files).

[mini.json](https://github.com/user-attachments/files/20535145/mini.json)

I've also found a number of issues which describe a similar problem:

[7569](https://github.com/huggingface/datasets/issues/7569) (Open)
[7137](https://github.com/huggingface/datasets/issues/7137) (Open)
[7501](https://github.com/huggingface/datasets/issues/7501) (Closed)
[2434](https://github.com/huggingface/datasets/issues/2434) (Closed)

The closed issues don't really address the problem (IMO). [7501](https://github.com/huggingface/datasets/issues/7501) provides a workaround (using a Python list instead of `Sequence`), but it seem precarious. 
2025-05-31T18:19:21Z
https://github.com/huggingface/datasets/issues/7590#issuecomment-2925533927
https://api.github.com/repos/huggingface/datasets/issues/7590
IC_kwDODunzps6uYBrn
https://api.github.com/repos/huggingface/datasets/issues/comments/2925533927/reactions
2025-05-31T18:21:03Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2925533927
https://avatars.githubusercontent.com/u/183279820?v=4
https://api.github.com/users/AHS-uni/events{/privacy}
https://api.github.com/users/AHS-uni/followers
https://api.github.com/users/AHS-uni/following{/other_user}
https://api.github.com/users/AHS-uni/gists{/gist_id}
https://github.com/AHS-uni
AHS-uni
U_kgDOCuygzA
https://api.github.com/users/AHS-uni/orgs
https://api.github.com/users/AHS-uni/received_events
https://api.github.com/users/AHS-uni/repos
https://api.github.com/users/AHS-uni/starred{/owner}{/repo}
https://api.github.com/users/AHS-uni/subscriptions
User
https://api.github.com/users/AHS-uni
public
MEMBER
Hi ! `Sequence({...})` corresponds to a struct of lists ([docs](https://huggingface.co/docs/datasets/v3.6.0/en/package_reference/main_classes#datasets.Features)). This come from Tensorflow Datasets.

If you want to use a list of structs, you should use `[{...}]`, e.g.

```python
item = {
    "id":   Value("string"),
    "data": Value("string"),
}

features = Features({
    "list": [item],
})
```
2025-06-04T13:12:34Z
https://github.com/huggingface/datasets/issues/7590#issuecomment-2939991397
https://api.github.com/repos/huggingface/datasets/issues/7590
IC_kwDODunzps6vPLVl
https://api.github.com/repos/huggingface/datasets/issues/comments/2939991397/reactions
2025-06-04T13:13:07Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2939991397
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
@lhoestq Thanks for your explanation, which helps me understand the logic behind. But I'm confused how to define that in `README.md`?

My jsonl data is: 
```
{"answers": [{"text": "text1", "label": "label1"}, {"text": "text2", "label": "label2"},]}
{"answers": [{"text": "text1", "label": "label1"}, {"text": "text2", "label": "label2"},]}
...
```

My README.md look like
```
dataset_info:
- config_name: default
  features:
  - name: answers
    sequence:
    - name: text
      dtype: string
    - name: label
      dtype: string
```
I understand `sequence` here is not correct, but what's the correct format? I tried following (`sequence -> dtype`)and seems not the case:
```
dataset_info:
- config_name: default
  features:
  - name: answers
    dtype:
    - name: text
      sequence: string
    - name: label
      sequence: string
```
2025-07-03T18:34:32Z
https://github.com/huggingface/datasets/issues/7590#issuecomment-3033191683
https://api.github.com/repos/huggingface/datasets/issues/7590
IC_kwDODunzps60ytUD
https://api.github.com/repos/huggingface/datasets/issues/comments/3033191683/reactions
2025-07-03T18:34:32Z
https://api.github.com/repos/huggingface/datasets/issues/comments/3033191683
https://avatars.githubusercontent.com/u/13214530?v=4
https://api.github.com/users/ain-soph/events{/privacy}
https://api.github.com/users/ain-soph/followers
https://api.github.com/users/ain-soph/following{/other_user}
https://api.github.com/users/ain-soph/gists{/gist_id}
https://github.com/ain-soph
ain-soph
MDQ6VXNlcjEzMjE0NTMw
https://api.github.com/users/ain-soph/orgs
https://api.github.com/users/ain-soph/received_events
https://api.github.com/users/ain-soph/repos
https://api.github.com/users/ain-soph/starred{/owner}{/repo}
https://api.github.com/users/ain-soph/subscriptions
User
https://api.github.com/users/ain-soph
public
MEMBER
The `List` type which doesn't have the weird dict behavior of `Sequence` has been added for `datasets` 4.0 (to be released next week). Feel free to install `datasets` from source to try it out :)
EDIT: it's out !

You can fix the issue using `List` instead of `Sequence`, e.g. in the case of the original post:

```python
# Feature spec with List of structs
item = {
    "id":   Value("string"),
    "data": Value("string"),
}

features = Features({
    "list": List(item),
})
```

for which the README.md is

```yaml
dataset_info:
- config_name: default
  features:
  - name: list
    list:
    - name: id
      dtype: string
    - name: data
      dtype: string
```
2025-07-04T11:08:43Z
https://github.com/huggingface/datasets/issues/7590#issuecomment-3035647354
https://api.github.com/repos/huggingface/datasets/issues/7590
IC_kwDODunzps608E16
https://api.github.com/repos/huggingface/datasets/issues/comments/3035647354/reactions
2025-07-10T08:39:24Z
https://api.github.com/repos/huggingface/datasets/issues/comments/3035647354
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
@lhoestq Thanks! I didn't realize there is a `list` keyword I could use. I thought I had to use `dtype` or something. Hope there could be better documentation on the `README.md` formats. I've closed my issue #7137 
2025-07-04T19:48:58Z
https://github.com/huggingface/datasets/issues/7590#issuecomment-3037148190
https://api.github.com/repos/huggingface/datasets/issues/7590
IC_kwDODunzps61BzQe
https://api.github.com/repos/huggingface/datasets/issues/comments/3037148190/reactions
2025-07-04T19:51:38Z
https://api.github.com/repos/huggingface/datasets/issues/comments/3037148190
https://avatars.githubusercontent.com/u/13214530?v=4
https://api.github.com/users/ain-soph/events{/privacy}
https://api.github.com/users/ain-soph/followers
https://api.github.com/users/ain-soph/following{/other_user}
https://api.github.com/users/ain-soph/gists{/gist_id}
https://github.com/ain-soph
ain-soph
MDQ6VXNlcjEzMjE0NTMw
https://api.github.com/users/ain-soph/orgs
https://api.github.com/users/ain-soph/received_events
https://api.github.com/users/ain-soph/repos
https://api.github.com/users/ain-soph/starred{/owner}{/repo}
https://api.github.com/users/ain-soph/subscriptions
User
https://api.github.com/users/ain-soph
public
### Description

When loading a dataset with a field declared as a list of structs using `Sequence(Features(...))`, `load_dataset` incorrectly infers the field as a plain `struct<...>` instead of a `list<struct<...>>`. This leads to the following error:

```
ArrowNotImplementedError: Unsupported cast from list<item: struct<id: string, data: string>> to struct using function cast_struct
```

This occurs even when the `features` schema is explicitly provided and the dataset format supports nested structures natively (e.g., JSON, JSONL).

---

###  Minimal Reproduction

[Colab Link.](https://colab.research.google.com/drive/1FZPQy6TP3jVd4B3mYKyfQaWNuOAvljUq?usp=sharing)

#### Dataset

```python
data = [
    {
        "list": [
            {"id": "example1", "data": "text"},
        ]
    },
]
```

#### Schema

```python
from datasets import Features, Sequence, Value

item = Features({
    "id":   Value("string"),
    "data": Value("string"),
})

features = Features({
    "list": Sequence(item),
})
```

---

###  Tested File Formats

The same schema was tested across different formats:

| Format    | Method                      | Result              |
| --------- | --------------------------- | ------------------- |
| JSONL     | `load_dataset("json", ...)` |  Arrow cast error  |
| JSON      | `load_dataset("json", ...)` |  Arrow cast error  |
| In-memory | `Dataset.from_list(...)`    | Works as expected |

The issue seems not to be in the schema or the data, but in how `load_dataset()` handles the `Sequence(Features(...))` pattern when parsing from files (specifically JSON and JSONL).

---

### Expected Behavior

If `features` is explicitly defined as:

```python
Features({"list": Sequence(Features({...}))})
```

Then the data should load correctly across all backends — including from JSON and JSONL — without any Arrow casting errors. This works correctly when loading from memory via `Dataset.from_list`.

---

###  Environment

* `datasets`: 3.6.0
* `pyarrow`: 20.0.0
* Python: 3.12.10
* OS: Ubuntu 24.04.2 LTS
* Notebook: \[Colab test notebook available]

---
