Question: Is there any way for uploading a large image dataset?
NONE
```
import pandas as pd
from datasets import Dataset, Image

# Read the CSV file
data = pd.read_csv("XXXX.csv")

# Create a Hugging Face Dataset
dataset = Dataset.from_pandas(data)
dataset = dataset.cast_column("file_name", Image())

# Upload to Hugging Face Hub (make sure authentication is set up)
dataset.push_to_hub("XXXXX"")
```

stuck in "Casting the dataset
![截屏2024-05-02 11 44 50](https://github.com/huggingface/datasets/assets/48406770/dc012dc5-16f6-4fd5-9e02-1b705c552c5b)
"

2024-05-02T03:44:58Z
https://github.com/huggingface/datasets/issues/6686#issuecomment-2089477693
https://api.github.com/repos/huggingface/datasets/issues/6686
IC_kwDODunzps58iuY9
https://api.github.com/repos/huggingface/datasets/issues/comments/2089477693/reactions
2024-05-02T03:44:58Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2089477693
https://avatars.githubusercontent.com/u/48406770?v=4
https://api.github.com/users/dirtycomputer/events{/privacy}
https://api.github.com/users/dirtycomputer/followers
https://api.github.com/users/dirtycomputer/following{/other_user}
https://api.github.com/users/dirtycomputer/gists{/gist_id}
https://github.com/dirtycomputer
dirtycomputer
MDQ6VXNlcjQ4NDA2Nzcw
https://api.github.com/users/dirtycomputer/orgs
https://api.github.com/users/dirtycomputer/received_events
https://api.github.com/users/dirtycomputer/repos
https://api.github.com/users/dirtycomputer/starred{/owner}{/repo}
https://api.github.com/users/dirtycomputer/subscriptions
User
https://api.github.com/users/dirtycomputer
public
I am uploading an image dataset like this:
```
dataset = load_dataset(
    "json",
    data_files={"train": "data/custom_dataset/train.json", "validation": "data/custom_dataset/val.json"},
)
dataset = dataset.cast_column("images", Sequence(Image()))
dataset.push_to_hub("StanfordAIMI/custom_dataset", max_shard_size="1GB")
```
where it takes a long time in the `Map` process. Do you think I can use multi-processing to map all the image data to the memory first? For the `Map()` function, I can set `num_proc`. But for `push_to_hub` and `cast_column`, I can not find it.

Thanks in advance!

Best,