IndexError: Invalid key: 14 is out of bounds for size 0
MEMBER
Hi ! Could you check the length of the `self.dataset` object (i.e. the Dataset object passed to the data loader) ? It looks like the dataset is empty.
Not sure why the SWA optimizer would cause this though.
2021-10-04T14:30:19Z
https://github.com/huggingface/datasets/issues/2988#issuecomment-933544418
https://api.github.com/repos/huggingface/datasets/issues/2988
IC_kwDODunzps43pMHi
https://api.github.com/repos/huggingface/datasets/issues/comments/933544418/reactions
2021-10-04T14:30:51Z
https://api.github.com/repos/huggingface/datasets/issues/comments/933544418
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
Any updates on this? 
The same error occurred to me too when running `cardiffnlp/twitter-roberta-base-sentiment` on a custom dataset. This happened when I tried to do `model = torch.nn.DataParallel(model, device_ids=[0, 1, 2, 3])` without using sagemaker distribution. 
Python: 3.6.13
datasets: 1.6.2
2021-12-07T03:27:34Z
https://github.com/huggingface/datasets/issues/2988#issuecomment-987534913
https://api.github.com/repos/huggingface/datasets/issues/2988
IC_kwDODunzps463JZB
https://api.github.com/repos/huggingface/datasets/issues/comments/987534913/reactions
2021-12-08T14:54:23Z
https://api.github.com/repos/huggingface/datasets/issues/comments/987534913
https://avatars.githubusercontent.com/u/12865645?v=4
https://api.github.com/users/ruisi-su/events{/privacy}
https://api.github.com/users/ruisi-su/followers
https://api.github.com/users/ruisi-su/following{/other_user}
https://api.github.com/users/ruisi-su/gists{/gist_id}
https://github.com/ruisi-su
ruisi-su
MDQ6VXNlcjEyODY1NjQ1
https://api.github.com/users/ruisi-su/orgs
https://api.github.com/users/ruisi-su/received_events
https://api.github.com/users/ruisi-su/repos
https://api.github.com/users/ruisi-su/starred{/owner}{/repo}
https://api.github.com/users/ruisi-su/subscriptions
User
https://api.github.com/users/ruisi-su
public
MEMBER
Hi @ruisi-su, do you have this issue while using SWA as well, or just data parallel ?

If you have a code example to reproduce this issue it would also be helpful
2021-12-13T10:39:05Z
https://github.com/huggingface/datasets/issues/2988#issuecomment-992329633
https://api.github.com/repos/huggingface/datasets/issues/2988
IC_kwDODunzps47Jb-h
https://api.github.com/repos/huggingface/datasets/issues/comments/992329633/reactions
2021-12-13T10:39:05Z
https://api.github.com/repos/huggingface/datasets/issues/comments/992329633
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
@lhoestq I had this issue without SWA. I followed [this](https://github.com/huggingface/notebooks/blob/master/sagemaker/03_distributed_training_data_parallelism/sagemaker-notebook.ipynb) notebook to utilize multiple gpus on the [roberta-base](https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment) model. This tutorial could only work if I am on `ml.p3.16xlarge`, which I don't have access to. So I tried using just `model = torch.nn.DataParallel(model, device_ids=[0, 1, 2, 3]` before calling `trainer.fit()`. But maybe this is not the right way to do distributed training. I can provide a code example if that will be more helpful.
2021-12-13T14:29:36Z
https://github.com/huggingface/datasets/issues/2988#issuecomment-992536863
https://api.github.com/repos/huggingface/datasets/issues/2988
IC_kwDODunzps47KOkf
https://api.github.com/repos/huggingface/datasets/issues/comments/992536863/reactions
2021-12-13T14:29:36Z
https://api.github.com/repos/huggingface/datasets/issues/comments/992536863
https://avatars.githubusercontent.com/u/12865645?v=4
https://api.github.com/users/ruisi-su/events{/privacy}
https://api.github.com/users/ruisi-su/followers
https://api.github.com/users/ruisi-su/following{/other_user}
https://api.github.com/users/ruisi-su/gists{/gist_id}
https://github.com/ruisi-su
ruisi-su
MDQ6VXNlcjEyODY1NjQ1
https://api.github.com/users/ruisi-su/orgs
https://api.github.com/users/ruisi-su/received_events
https://api.github.com/users/ruisi-su/repos
https://api.github.com/users/ruisi-su/starred{/owner}{/repo}
https://api.github.com/users/ruisi-su/subscriptions
User
https://api.github.com/users/ruisi-su
public
MEMBER
It might be an issue with old versions of `datasets`, can you try updating `datasets` ?
2021-12-13T17:24:07Z
https://github.com/huggingface/datasets/issues/2988#issuecomment-992701717
https://api.github.com/repos/huggingface/datasets/issues/2988
IC_kwDODunzps47K20V
https://api.github.com/repos/huggingface/datasets/issues/comments/992701717/reactions
2021-12-13T17:24:07Z
https://api.github.com/repos/huggingface/datasets/issues/comments/992701717
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
FYI I encountered the exact same error using the latest versions of `datasets`, `transformers` and `pyarrow`, without using any kind of SWA or dataparallel: 

```
# packages in environment at C:\Users\zhang\mambaforge:
#
# Name                    Version                   Build  Channel
cudatoolkit               11.0.3               h3f58a73_9    https://mirrors.ustc.edu.cn/anaconda/cloud/conda-forge
datasets                  1.17.0                   pypi_0    pypi
pyarrow                   6.0.1                    pypi_0    pypi
pytorch                   1.7.1           py3.9_cuda110_cudnn8_0    pytorch
tornado                   6.1              py39hb82d6ee_2    https://mirrors.ustc.edu.cn/anaconda/cloud/conda-forge
```

```
> python --version
> 3.9.7
```
2022-01-05T06:38:06Z
https://github.com/huggingface/datasets/issues/2988#issuecomment-1005421116
https://api.github.com/repos/huggingface/datasets/issues/2988
IC_kwDODunzps477YI8
https://api.github.com/repos/huggingface/datasets/issues/comments/1005421116/reactions
2022-01-05T06:38:06Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1005421116
https://avatars.githubusercontent.com/u/19884169?v=4
https://api.github.com/users/z-q-y/events{/privacy}
https://api.github.com/users/z-q-y/followers
https://api.github.com/users/z-q-y/following{/other_user}
https://api.github.com/users/z-q-y/gists{/gist_id}
https://github.com/z-q-y
z-q-y
MDQ6VXNlcjE5ODg0MTY5
https://api.github.com/users/z-q-y/orgs
https://api.github.com/users/z-q-y/received_events
https://api.github.com/users/z-q-y/repos
https://api.github.com/users/z-q-y/starred{/owner}{/repo}
https://api.github.com/users/z-q-y/subscriptions
User
https://api.github.com/users/z-q-y
public
NONE
Same error here! Datasets version `1.18.3` freshly updated.

`IndexError: Invalid key: 90 is out of bounds for size 0`

My task is finetuning the model for token classification.

**Solved**: I make a mistake while updating the dataset during the map, you should check that you return the correct values.

2022-03-03T21:20:30Z
https://github.com/huggingface/datasets/issues/2988#issuecomment-1058505614
https://api.github.com/repos/huggingface/datasets/issues/2988
IC_kwDODunzps4_F4OO
https://api.github.com/repos/huggingface/datasets/issues/comments/1058505614/reactions
2022-03-03T21:33:58Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1058505614
https://avatars.githubusercontent.com/u/10062811?v=4
https://api.github.com/users/MorenoLaQuatra/events{/privacy}
https://api.github.com/users/MorenoLaQuatra/followers
https://api.github.com/users/MorenoLaQuatra/following{/other_user}
https://api.github.com/users/MorenoLaQuatra/gists{/gist_id}
https://github.com/MorenoLaQuatra
MorenoLaQuatra
MDQ6VXNlcjEwMDYyODEx
https://api.github.com/users/MorenoLaQuatra/orgs
https://api.github.com/users/MorenoLaQuatra/received_events
https://api.github.com/users/MorenoLaQuatra/repos
https://api.github.com/users/MorenoLaQuatra/starred{/owner}{/repo}
https://api.github.com/users/MorenoLaQuatra/subscriptions
User
https://api.github.com/users/MorenoLaQuatra
public
MEMBER
cc @sgugger This probably comes from the `Trainer` removing all the columns of a dataset, do you think we can improve the error message in this case ?
2022-03-04T15:25:14Z
https://github.com/huggingface/datasets/issues/2988#issuecomment-1059260137
https://api.github.com/repos/huggingface/datasets/issues/2988
IC_kwDODunzps4_Iwbp
https://api.github.com/repos/huggingface/datasets/issues/comments/1059260137/reactions
2022-03-04T15:25:14Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1059260137
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
The `Trainer` clearly logs when it removes columns in the dataset. I'm not too sure of where the bug appears as I haven't seen a clear reproducer. Happy to display a more helpful error message, but I'd need a reproducer to see what the exact problem is to design the right test and warning :-) 
2022-03-04T15:35:41Z
https://github.com/huggingface/datasets/issues/2988#issuecomment-1059268627
https://api.github.com/repos/huggingface/datasets/issues/2988
IC_kwDODunzps4_IygT
https://api.github.com/repos/huggingface/datasets/issues/comments/1059268627/reactions
2022-03-04T15:35:41Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1059268627
https://avatars.githubusercontent.com/u/35901082?v=4
https://api.github.com/users/sgugger/events{/privacy}
https://api.github.com/users/sgugger/followers
https://api.github.com/users/sgugger/following{/other_user}
https://api.github.com/users/sgugger/gists{/gist_id}
https://github.com/sgugger
sgugger
MDQ6VXNlcjM1OTAxMDgy
https://api.github.com/users/sgugger/orgs
https://api.github.com/users/sgugger/received_events
https://api.github.com/users/sgugger/repos
https://api.github.com/users/sgugger/starred{/owner}{/repo}
https://api.github.com/users/sgugger/subscriptions
User
https://api.github.com/users/sgugger
public
NONE
Well, if I can try to suggest how to reproduce, please try by do not returning any updated content in the map function used to tokenize input (e.g., in TokenClassification). I can leave here my wrong version for reference:

```python
def preprocess_function(examples):

    text = examples["text"]
    
    inputs = tokenizer(
        text,
        max_length=512,
        truncation="only_second",
        return_offsets_mapping=True,
        padding="max_length",
    )

    offset_mapping = inputs.pop("offset_mapping")
    # ... processing code

    inputs["labels"] = label_ids
    #return inputs
    
train_ds = train_ds.map(preprocess_function, batched=False)
test_ds = test_ds.map(preprocess_function, batched=False)
eval_ds = eval_ds.map(preprocess_function, batched=False)
```

Of course, returning inputs solved the problem. As suggestion, a possible error message could display "IndexError: the `key` required by trainer are not found in the dataset" (just an hypothesis, I think there could be something better). 

Please tell me if you need more details to reproduce, glad to help!
2022-03-04T21:53:32Z
https://github.com/huggingface/datasets/issues/2988#issuecomment-1059558586
https://api.github.com/repos/huggingface/datasets/issues/2988
IC_kwDODunzps4_J5S6
https://api.github.com/repos/huggingface/datasets/issues/comments/1059558586/reactions
2022-03-04T21:55:07Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1059558586
https://avatars.githubusercontent.com/u/10062811?v=4
https://api.github.com/users/MorenoLaQuatra/events{/privacy}
https://api.github.com/users/MorenoLaQuatra/followers
https://api.github.com/users/MorenoLaQuatra/following{/other_user}
https://api.github.com/users/MorenoLaQuatra/gists{/gist_id}
https://github.com/MorenoLaQuatra
MorenoLaQuatra
MDQ6VXNlcjEwMDYyODEx
https://api.github.com/users/MorenoLaQuatra/orgs
https://api.github.com/users/MorenoLaQuatra/received_events
https://api.github.com/users/MorenoLaQuatra/repos
https://api.github.com/users/MorenoLaQuatra/starred{/owner}{/repo}
https://api.github.com/users/MorenoLaQuatra/subscriptions
User
https://api.github.com/users/MorenoLaQuatra
public
CONTRIBUTOR
That's the thing though. The `Trainer` has no idea which inputs are required or not since all models can have different kinds of inputs, and it can work for models outside of the Transformers library. I can add a clear error message if I get an empty batch, as this is easy to detect, but that's pretty much it.
2022-03-07T16:21:51Z
https://github.com/huggingface/datasets/issues/2988#issuecomment-1060873277
https://api.github.com/repos/huggingface/datasets/issues/2988
IC_kwDODunzps4_O6Q9
https://api.github.com/repos/huggingface/datasets/issues/comments/1060873277/reactions
2022-03-07T16:21:51Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1060873277
https://avatars.githubusercontent.com/u/35901082?v=4
https://api.github.com/users/sgugger/events{/privacy}
https://api.github.com/users/sgugger/followers
https://api.github.com/users/sgugger/following{/other_user}
https://api.github.com/users/sgugger/gists{/gist_id}
https://github.com/sgugger
sgugger
MDQ6VXNlcjM1OTAxMDgy
https://api.github.com/users/sgugger/orgs
https://api.github.com/users/sgugger/received_events
https://api.github.com/users/sgugger/repos
https://api.github.com/users/sgugger/starred{/owner}{/repo}
https://api.github.com/users/sgugger/subscriptions
User
https://api.github.com/users/sgugger
public
NONE
I think that it could be enough to ease the identification of the problem.
2022-03-07T18:07:18Z
https://github.com/huggingface/datasets/issues/2988#issuecomment-1060976419
https://api.github.com/repos/huggingface/datasets/issues/2988
IC_kwDODunzps4_PTcj
https://api.github.com/repos/huggingface/datasets/issues/comments/1060976419/reactions
2022-03-07T18:07:18Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1060976419
https://avatars.githubusercontent.com/u/10062811?v=4
https://api.github.com/users/MorenoLaQuatra/events{/privacy}
https://api.github.com/users/MorenoLaQuatra/followers
https://api.github.com/users/MorenoLaQuatra/following{/other_user}
https://api.github.com/users/MorenoLaQuatra/gists{/gist_id}
https://github.com/MorenoLaQuatra
MorenoLaQuatra
MDQ6VXNlcjEwMDYyODEx
https://api.github.com/users/MorenoLaQuatra/orgs
https://api.github.com/users/MorenoLaQuatra/received_events
https://api.github.com/users/MorenoLaQuatra/repos
https://api.github.com/users/MorenoLaQuatra/starred{/owner}{/repo}
https://api.github.com/users/MorenoLaQuatra/subscriptions
User
https://api.github.com/users/MorenoLaQuatra
public
CONTRIBUTOR
Done in [this commit](https://github.com/huggingface/transformers/commit/c87cfd653c4de3d4743a9ae09d749282d94d5829)
2022-03-07T18:43:41Z
https://github.com/huggingface/datasets/issues/2988#issuecomment-1061010247
https://api.github.com/repos/huggingface/datasets/issues/2988
IC_kwDODunzps4_PbtH
https://api.github.com/repos/huggingface/datasets/issues/comments/1061010247/reactions
2022-03-07T18:43:41Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1061010247
https://avatars.githubusercontent.com/u/35901082?v=4
https://api.github.com/users/sgugger/events{/privacy}
https://api.github.com/users/sgugger/followers
https://api.github.com/users/sgugger/following{/other_user}
https://api.github.com/users/sgugger/gists{/gist_id}
https://github.com/sgugger
sgugger
MDQ6VXNlcjM1OTAxMDgy
https://api.github.com/users/sgugger/orgs
https://api.github.com/users/sgugger/received_events
https://api.github.com/users/sgugger/repos
https://api.github.com/users/sgugger/starred{/owner}{/repo}
https://api.github.com/users/sgugger/subscriptions
User
https://api.github.com/users/sgugger
public
## Describe the bug
A clear and concise description of what the bug is.

Hi. I am trying to implement stochastic weighted averaging optimizer with transformer library as described here https://pytorch.org/blog/pytorch-1.6-now-includes-stochastic-weight-averaging/ , for this I am using a run_clm.py codes which is working fine before adding SWA optimizer, the moment I modify the model with `swa_model = AveragedModel(model)` in this script, I am getting the below error,  since I am NOT touching the dataloader part, I am confused why this is occurring, I very much appreciate your opinion on this @lhoestq 

## Steps to reproduce the bug
```
Traceback (most recent call last):
  File "run_clm.py", line 723, in <module>
    main()
  File "run_clm.py", line 669, in main
    train_result = trainer.train(resume_from_checkpoint=checkpoint)
  File "/user/dara/libs/anaconda3/envs/success/lib/python3.7/site-packages/transformers/trainer.py", line 1258, in train
    for step, inputs in enumerate(epoch_iterator):
  File "/user/dara/libs/anaconda3/envs/success/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 435, in __next__
    data = self._next_data()
  File "/user/dara/libs/anaconda3/envs/success/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 475, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/user/dara/libs/anaconda3/envs/success/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/user/dara/libs/anaconda3/envs/success/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/user/dara/libs/anaconda3/envs/success/lib/python3.7/site-packages/datasets/arrow_dataset.py", line 1530, in __getitem__
    format_kwargs=self._format_kwargs,
  File "/user/dara/libs/anaconda3/envs/success/lib/python3.7/site-packages/datasets/arrow_dataset.py", line 1517, in _getitem
    pa_subtable = query_table(self._data, key, indices=self._indices if self._indices is not None else None)
  File "/user/dara/libs/anaconda3/envs/success/lib/python3.7/site-packages/datasets/formatting/formatting.py", line 368, in query_table
    _check_valid_index_key(key, size)
  File "/user/dara/libs/anaconda3/envs/success/lib/python3.7/site-packages/datasets/formatting/formatting.py", line 311, in _check_valid_index_key
    raise IndexError(f"Invalid key: {key} is out of bounds for size {size}")
IndexError: Invalid key: 14 is out of bounds for size 0
```



## Expected results
not getting the index error

## Actual results
Please see the above

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: datasets                  1.12.1 
- Platform: linux
- Python version: 3.7.11 
- PyArrow version:  5.0.0 



