Error when loading a big local json file
NONE
I get similar when dealing with a large jsonl file (6k lines), 

> TypeError: Couldn't cast array of type timestamp[us] to null

Yet when I split it into 1k lines, files, load_dataset works fine!

https://github.com/huggingface/course/issues/692


2024-03-15T22:18:20Z
https://github.com/huggingface/datasets/issues/6656#issuecomment-2000576331
https://api.github.com/repos/huggingface/datasets/issues/6656
IC_kwDODunzps53Pl9L
https://api.github.com/repos/huggingface/datasets/issues/comments/2000576331/reactions
2024-03-15T22:18:20Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2000576331
https://avatars.githubusercontent.com/u/4273247?v=4
https://api.github.com/users/fancellu/events{/privacy}
https://api.github.com/users/fancellu/followers
https://api.github.com/users/fancellu/following{/other_user}
https://api.github.com/users/fancellu/gists{/gist_id}
https://github.com/fancellu
fancellu
MDQ6VXNlcjQyNzMyNDc=
https://api.github.com/users/fancellu/orgs
https://api.github.com/users/fancellu/received_events
https://api.github.com/users/fancellu/repos
https://api.github.com/users/fancellu/starred{/owner}{/repo}
https://api.github.com/users/fancellu/subscriptions
User
https://api.github.com/users/fancellu
public
NONE
What's the proposed solution? :-)
2024-11-29T10:06:56Z
https://github.com/huggingface/datasets/issues/6656#issuecomment-2507480436
https://api.github.com/repos/huggingface/datasets/issues/6656
IC_kwDODunzps6VdR10
https://api.github.com/repos/huggingface/datasets/issues/comments/2507480436/reactions
2024-11-29T10:06:56Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2507480436
https://avatars.githubusercontent.com/u/36575651?v=4
https://api.github.com/users/LeonardoEmili/events{/privacy}
https://api.github.com/users/LeonardoEmili/followers
https://api.github.com/users/LeonardoEmili/following{/other_user}
https://api.github.com/users/LeonardoEmili/gists{/gist_id}
https://github.com/LeonardoEmili
LeonardoEmili
MDQ6VXNlcjM2NTc1NjUx
https://api.github.com/users/LeonardoEmili/orgs
https://api.github.com/users/LeonardoEmili/received_events
https://api.github.com/users/LeonardoEmili/repos
https://api.github.com/users/LeonardoEmili/starred{/owner}{/repo}
https://api.github.com/users/LeonardoEmili/subscriptions
User
https://api.github.com/users/LeonardoEmili
public
### Describe the bug

When trying to load big json files from a local directory, `load_dataset` throws the following error

```
Traceback (most recent call last):
  File "/miniconda3/envs/conda-env/lib/python3.10/site-packages/datasets/builder.py", line 1989, in _prepare_split_single
    writer.write_table(table)
  File "miniconda3/envs/conda-env/lib/python3.10/site-packages/datasets/arrow_writer.py", line 573, in write_table
    pa_table = pa_table.combine_chunks()
  File "pyarrow/table.pxi", line 3638, in pyarrow.lib.Table.combine_chunks
  File "pyarrow/error.pxi", line 154, in pyarrow.lib.pyarrow_internal_check_status
  File "pyarrow/error.pxi", line 91, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: offset overflow while concatenating arrays
```

### Steps to reproduce the bug

1. Download a big file, e.g. `https://dl.fbaipublicfiles.com/dpr/data/retriever/biencoder-nq-train.json.gz`
2. Load it like `data = load_dataset("json", data_files=["nq-train.json"], split="train")`

```python
from datasets import load_dataset

data = load_dataset("json", data_files=["nq-train.json"], split="train")
```

A similarly formatted but smaller file, e.g. e.g. `https://dl.fbaipublicfiles.com/dpr/data/retriever/biencoder-nq-dev.json.gz` is loaded without issues

```python
from datasets import load_dataset

data = load_dataset("json", data_files=["nq-dev.json"], split="train")
```

### Expected behavior

It should load normally

### Environment info

- `datasets` version: 2.16.1
- Platform: Linux-5.18.10-76051810-generic-x86_64-with-glibc2.31
- Python version: 3.10.13
- `huggingface_hub` version: 0.20.3
- PyArrow version: 15.0.0
- Pandas version: 2.2.0
- `fsspec` version: 2023.10.0