PyTorch BatchSampler still loads from Dataset one-by-one
MEMBER
This change seems to come from a few months ago in the PyTorch side. That's good news and it means we may not need to pass a batch_sampler as soon as we add `Dataset.__getitems__` to get the optimal speed :)

Thanks for reporting ! Would you like to open a PR to add `__getitems__` and remove this outdated documentation ?
2023-02-07T15:45:10Z
https://github.com/huggingface/datasets/issues/5505#issuecomment-1420991056
https://api.github.com/repos/huggingface/datasets/issues/5505
IC_kwDODunzps5UsppQ
https://api.github.com/repos/huggingface/datasets/issues/comments/1420991056/reactions
2023-02-07T15:45:10Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1420991056
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
Yeah I figured this was the sort of thing that probably once worked. I can confirm that you no longer need the batch sampler, just `batch_size=n` in the `DataLoader`.

I'll pass on the PR, I'm flat out right now, sorry.
2023-02-07T19:37:04Z
https://github.com/huggingface/datasets/issues/5505#issuecomment-1421340121
https://api.github.com/repos/huggingface/datasets/issues/5505
IC_kwDODunzps5Ut-3Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1421340121/reactions
2023-02-07T19:37:04Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1421340121
https://avatars.githubusercontent.com/u/4443482?v=4
https://api.github.com/users/davidgilbertson/events{/privacy}
https://api.github.com/users/davidgilbertson/followers
https://api.github.com/users/davidgilbertson/following{/other_user}
https://api.github.com/users/davidgilbertson/gists{/gist_id}
https://github.com/davidgilbertson
davidgilbertson
MDQ6VXNlcjQ0NDM0ODI=
https://api.github.com/users/davidgilbertson/orgs
https://api.github.com/users/davidgilbertson/received_events
https://api.github.com/users/davidgilbertson/repos
https://api.github.com/users/davidgilbertson/starred{/owner}{/repo}
https://api.github.com/users/davidgilbertson/subscriptions
User
https://api.github.com/users/davidgilbertson
public
### Describe the bug

In [the docs here](https://huggingface.co/docs/datasets/use_with_pytorch#use-a-batchsampler), it mentions the issue of the Dataset being read one-by-one, then states that using a BatchSampler resolves the issue.

I'm not sure if this is a mistake in the docs or the code, but it seems that the only way for a Dataset to be passed a list of indexes  by PyTorch (instead of one index at a time) is to define a `__getitems__` method (note the plural) on the Dataset object, and since the HF Dataset doesn't have this, PyTorch executes [this line of code](https://github.com/pytorch/pytorch/blob/master/torch/utils/data/_utils/fetch.py#L58), reverting to fetching one-by-one.

### Steps to reproduce the bug

You can put a breakpoint in `Dataset.__getitem__()` or just print the args from there and see that it's called multiple times for a single `next(iter(dataloader))`, even when using the code from the docs:
```py
from torch.utils.data.sampler import BatchSampler, RandomSampler
batch_sampler = BatchSampler(RandomSampler(ds), batch_size=32, drop_last=False)
dataloader = DataLoader(ds, batch_sampler=batch_sampler)
```



### Expected behavior

The expected behaviour would be for it to fetch batches from the dataset, rather than one-by-one.

To demonstrate that there is room for improvement: once I have a HF dataset `ds`, if I just add this line:
```py
ds.__getitems__ = ds.__getitem__
```
...then the time taken to loop over the dataset improves considerably (for wikitext-103, from one minute to 13 seconds with batch size 32). Probably not a big deal in the grand scheme of things, but seems like an easy win.

### Environment info

- `datasets` version: 2.9.0
- Platform: Linux-5.10.102.1-microsoft-standard-WSL2-x86_64-with-glibc2.31
- Python version: 3.10.8
- PyArrow version: 10.0.1
- Pandas version: 1.5.3