Super slow iteration with trivial custom transform
NONE
Similar issue in text process 

```python

tokenizer=AutoTokenizer.from_pretrained(model_dir[args.model])
train_dataset=datasets.load_from_disk(dataset_dir[args.dataset],keep_in_memory=True)['train']
train_dataset=train_dataset.map(partial(dname2func[args.dataset],tokenizer=tokenizer),batched=True,num_proc =50,remove_columns=train_dataset.features.keys(),desc='tokenize',keep_in_memory=True)

```
After this train_dataset will be like
```python
Dataset({
    features: ['input_ids', 'labels'],
    num_rows: 51760
})
```
In which input_ids and labels are both List[int]
However, per iter on dataset cost 7.412479639053345s ……？
```python
for j in tqdm(range(len(train_dataset)),desc='first stage'):
    input_id,label=train_dataset['input_ids'][j],train_dataset['labels'][j]

``` 
2024-04-27T14:49:28Z
https://github.com/huggingface/datasets/issues/6833#issuecomment-2080828158
https://api.github.com/repos/huggingface/datasets/issues/6833
IC_kwDODunzps58Bur-
https://api.github.com/repos/huggingface/datasets/issues/comments/2080828158/reactions
2024-04-27T14:49:43Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2080828158
https://avatars.githubusercontent.com/u/88258534?v=4
https://api.github.com/users/rangehow/events{/privacy}
https://api.github.com/users/rangehow/followers
https://api.github.com/users/rangehow/following{/other_user}
https://api.github.com/users/rangehow/gists{/gist_id}
https://github.com/rangehow
rangehow
MDQ6VXNlcjg4MjU4NTM0
https://api.github.com/users/rangehow/orgs
https://api.github.com/users/rangehow/received_events
https://api.github.com/users/rangehow/repos
https://api.github.com/users/rangehow/starred{/owner}{/repo}
https://api.github.com/users/rangehow/subscriptions
User
https://api.github.com/users/rangehow
public
MEMBER
The transform currently replaces the numpy formatting.

So you're back to copying data to long python lists which is super slow.

It would be cool for the transform to not remove the formatting in this case, but this requires a few changes in the lib
2024-05-04T11:24:35Z
https://github.com/huggingface/datasets/issues/6833#issuecomment-2094127571
https://api.github.com/repos/huggingface/datasets/issues/6833
IC_kwDODunzps580dnT
https://api.github.com/repos/huggingface/datasets/issues/comments/2094127571/reactions
2024-05-04T11:24:35Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2094127571
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
This also (somewhat surprisingly) affects iterable datasets, making map very challenging to use for data with large arrays, unless there is some workaround?
2024-10-08T12:57:48Z
https://github.com/huggingface/datasets/issues/6833#issuecomment-2399776654
https://api.github.com/repos/huggingface/datasets/issues/6833
IC_kwDODunzps6PCa-O
https://api.github.com/repos/huggingface/datasets/issues/comments/2399776654/reactions
2024-10-08T12:58:32Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2399776654
https://avatars.githubusercontent.com/u/5719745?v=4
https://api.github.com/users/alex-hh/events{/privacy}
https://api.github.com/users/alex-hh/followers
https://api.github.com/users/alex-hh/following{/other_user}
https://api.github.com/users/alex-hh/gists{/gist_id}
https://github.com/alex-hh
alex-hh
MDQ6VXNlcjU3MTk3NDU=
https://api.github.com/users/alex-hh/orgs
https://api.github.com/users/alex-hh/received_events
https://api.github.com/users/alex-hh/repos
https://api.github.com/users/alex-hh/starred{/owner}{/repo}
https://api.github.com/users/alex-hh/subscriptions
User
https://api.github.com/users/alex-hh
public
MEMBER
For iterable datasets you should be able to do this without slow downs
```python
ds = ds.with_format("arrow").map(...)
```

I haven't tried with "numpy" though, maybe there is a step that does Arrow -> List -> NumPy instead of Arrow -> NumPy directly. If it's the case it would be cool to avoid that
2024-10-08T13:12:59Z
https://github.com/huggingface/datasets/issues/6833#issuecomment-2399812412
https://api.github.com/repos/huggingface/datasets/issues/6833
IC_kwDODunzps6PCjs8
https://api.github.com/repos/huggingface/datasets/issues/comments/2399812412/reactions
2024-10-08T13:12:59Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2399812412
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Thanks! this works for me

However, it raises an error if batched=False and map batch_size isn't explicitly set to 1 due to map's default batch_size affecting the batch size of the RebatchedArrowExamplesIterable - is this a bug?
2024-10-08T13:48:58Z
https://github.com/huggingface/datasets/issues/6833#issuecomment-2399903885
https://api.github.com/repos/huggingface/datasets/issues/6833
IC_kwDODunzps6PC6CN
https://api.github.com/repos/huggingface/datasets/issues/comments/2399903885/reactions
2024-10-08T13:49:22Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2399903885
https://avatars.githubusercontent.com/u/5719745?v=4
https://api.github.com/users/alex-hh/events{/privacy}
https://api.github.com/users/alex-hh/followers
https://api.github.com/users/alex-hh/following{/other_user}
https://api.github.com/users/alex-hh/gists{/gist_id}
https://github.com/alex-hh
alex-hh
MDQ6VXNlcjU3MTk3NDU=
https://api.github.com/users/alex-hh/orgs
https://api.github.com/users/alex-hh/received_events
https://api.github.com/users/alex-hh/repos
https://api.github.com/users/alex-hh/starred{/owner}{/repo}
https://api.github.com/users/alex-hh/subscriptions
User
https://api.github.com/users/alex-hh
public
MEMBER
Thanks for the fix @alex-hh !
2024-10-08T14:20:14Z
https://github.com/huggingface/datasets/issues/6833#issuecomment-2399993181
https://api.github.com/repos/huggingface/datasets/issues/6833
IC_kwDODunzps6PDP1d
https://api.github.com/repos/huggingface/datasets/issues/comments/2399993181/reactions
2024-10-08T14:20:22Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2399993181
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
opened a new issue for the numpy slowdown #7206 
2024-10-08T15:41:17Z
https://github.com/huggingface/datasets/issues/6833#issuecomment-2400200029
https://api.github.com/repos/huggingface/datasets/issues/6833
IC_kwDODunzps6PECVd
https://api.github.com/repos/huggingface/datasets/issues/comments/2400200029/reactions
2024-10-08T15:41:17Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2400200029
https://avatars.githubusercontent.com/u/5719745?v=4
https://api.github.com/users/alex-hh/events{/privacy}
https://api.github.com/users/alex-hh/followers
https://api.github.com/users/alex-hh/following{/other_user}
https://api.github.com/users/alex-hh/gists{/gist_id}
https://github.com/alex-hh
alex-hh
MDQ6VXNlcjU3MTk3NDU=
https://api.github.com/users/alex-hh/orgs
https://api.github.com/users/alex-hh/received_events
https://api.github.com/users/alex-hh/repos
https://api.github.com/users/alex-hh/starred{/owner}{/repo}
https://api.github.com/users/alex-hh/subscriptions
User
https://api.github.com/users/alex-hh
public
### Describe the bug

Dataset is 10X slower when applying trivial transforms:

```
import time
import numpy as np
from datasets import Dataset, Features, Array2D

a = np.zeros((800, 800))
a = np.stack([a] * 1000)
features = Features({"a": Array2D(shape=(800, 800), dtype="uint8")})

ds1 = Dataset.from_dict({"a": a}, features=features).with_format('numpy')

def transform(batch):
	return batch

ds2 = ds1.with_transform(transform)

%time sum(1 for _ in ds1)
%time sum(1 for _ in ds2)
```

```
CPU times: user 472 ms, sys: 319 ms, total: 791 ms
Wall time: 794 ms
CPU times: user 9.32 s, sys: 443 ms, total: 9.76 s
Wall time: 9.78 s
```

In my real code I'm using set_transform to apply some post-processing on-the-fly for the 2d array, but it significantly slows down the dataset even if the transform itself is trivial.

Related issue: https://github.com/huggingface/datasets/issues/5841

### Steps to reproduce the bug

Use code in the description to reproduce.

### Expected behavior

Trivial custom transform in the example should not slowdown the dataset iteration.

### Environment info

- `datasets` version: 2.18.0
- Platform: Linux-5.15.0-79-generic-x86_64-with-glibc2.35
- Python version: 3.11.4
- `huggingface_hub` version: 0.20.2
- PyArrow version: 15.0.0
- Pandas version: 1.5.3
- `fsspec` version: 2023.12.2