IndexError during training with Squad dataset and T5-small model
NONE
add remove_unused_columns=False to training_args
https://github.com/huggingface/datasets/issues/6535#issuecomment-1874024704
2024-06-18T08:06:13Z
https://github.com/huggingface/datasets/issues/6973#issuecomment-2175467364
https://api.github.com/repos/huggingface/datasets/issues/6973
IC_kwDODunzps6Bqv9k
https://api.github.com/repos/huggingface/datasets/issues/comments/2175467364/reactions
2024-06-18T08:06:43Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2175467364
https://avatars.githubusercontent.com/u/16257131?v=4
https://api.github.com/users/arthasking123/events{/privacy}
https://api.github.com/users/arthasking123/followers
https://api.github.com/users/arthasking123/following{/other_user}
https://api.github.com/users/arthasking123/gists{/gist_id}
https://github.com/arthasking123
arthasking123
MDQ6VXNlcjE2MjU3MTMx
https://api.github.com/users/arthasking123/orgs
https://api.github.com/users/arthasking123/received_events
https://api.github.com/users/arthasking123/repos
https://api.github.com/users/arthasking123/starred{/owner}{/repo}
https://api.github.com/users/arthasking123/subscriptions
User
https://api.github.com/users/arthasking123
public
MEMBER
Closing this issue because it was a reported and fixed in transformers.
2024-07-01T11:25:40Z
https://github.com/huggingface/datasets/issues/6973#issuecomment-2199892660
https://api.github.com/repos/huggingface/datasets/issues/6973
IC_kwDODunzps6DH7K0
https://api.github.com/repos/huggingface/datasets/issues/comments/2199892660/reactions
2024-07-01T11:25:40Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2199892660
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
### Describe the bug

I am encountering an IndexError while training a T5-small model on the Squad dataset using the transformers and datasets libraries. The error occurs even with a minimal reproducible example, suggesting a potential bug or incompatibility.

### Steps to reproduce the bug

1.Install the required libraries: !pip install transformers datasets
2.Run the following code:
!pip install transformers datasets

import torch
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM, TrainingArguments, Trainer, DataCollatorWithPadding

# Load a small, publicly available dataset
from datasets import load_dataset
dataset = load_dataset("squad", split="train[:100]") # Use a small subset for testing

# Load a pre-trained model and tokenizer
model_name = "t5-small"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)

# Define a basic data collator
data_collator = DataCollatorWithPadding(tokenizer=tokenizer)

# Define training arguments
training_args = TrainingArguments(
    output_dir="./results",
    per_device_train_batch_size=2,
    num_train_epochs=1,
)

# Create a trainer
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=dataset,
    data_collator=data_collator,
)

# Train the model
trainer.train()

### Expected behavior

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
[<ipython-input-23-f13a4b23c001>](https://localhost:8080/#) in <cell line: 34>()
     32 
     33 # Train the model
---> 34 trainer.train()

10 frames
[/usr/local/lib/python3.10/dist-packages/datasets/formatting/formatting.py](https://localhost:8080/#) in _check_valid_index_key(key, size)
    427     if isinstance(key, int):
    428         if (key < 0 and key + size < 0) or (key >= size):
--> 429             raise IndexError(f"Invalid key: {key} is out of bounds for size {size}")
    430         return
    431     elif isinstance(key, slice):

IndexError: Invalid key: 42 is out of bounds for size 0

### Environment info

transformers version:4.41.2
datasets version:1.18.4
Python version:3.10.12
