Dataset.map()'s fails at any value of parameter writer_batch_size 
NONE
Note that this same issue occurs even if i preprocess with the more default way of tokenizing that uses LayoutLMv2Processor's internal OCR:

```python
        feature_extractor = LayoutLMv2FeatureExtractor()
        tokenizer = LayoutLMv2Tokenizer.from_pretrained("microsoft/layoutlmv2-base-uncased")
        processor = LayoutLMv2Processor(feature_extractor, tokenizer)
        encoded_inputs = processor(images, padding="max_length", truncation=True)
        encoded_inputs["image"] = np.array(encoded_inputs["image"])
        encoded_inputs["label"] = examples['label_id']
```
2022-05-14T14:08:50Z
https://github.com/huggingface/datasets/issues/4349#issuecomment-1126723155
https://api.github.com/repos/huggingface/datasets/issues/4349
IC_kwDODunzps5DKG5T
https://api.github.com/repos/huggingface/datasets/issues/comments/1126723155/reactions
2022-05-14T14:08:50Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1126723155
https://avatars.githubusercontent.com/u/99206017?v=4
https://api.github.com/users/plamb-viso/events{/privacy}
https://api.github.com/users/plamb-viso/followers
https://api.github.com/users/plamb-viso/following{/other_user}
https://api.github.com/users/plamb-viso/gists{/gist_id}
https://github.com/plamb-viso
plamb-viso
U_kgDOBenDgQ
https://api.github.com/users/plamb-viso/orgs
https://api.github.com/users/plamb-viso/received_events
https://api.github.com/users/plamb-viso/repos
https://api.github.com/users/plamb-viso/starred{/owner}{/repo}
https://api.github.com/users/plamb-viso/subscriptions
User
https://api.github.com/users/plamb-viso
public
NONE
Wanted to make sure anyone that finds this also finds my other report: https://github.com/huggingface/datasets/issues/4352
2022-05-14T18:02:21Z
https://github.com/huggingface/datasets/issues/4349#issuecomment-1126786079
https://api.github.com/repos/huggingface/datasets/issues/4349
IC_kwDODunzps5DKWQf
https://api.github.com/repos/huggingface/datasets/issues/comments/1126786079/reactions
2022-05-14T18:02:21Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1126786079
https://avatars.githubusercontent.com/u/99206017?v=4
https://api.github.com/users/plamb-viso/events{/privacy}
https://api.github.com/users/plamb-viso/followers
https://api.github.com/users/plamb-viso/following{/other_user}
https://api.github.com/users/plamb-viso/gists{/gist_id}
https://github.com/plamb-viso
plamb-viso
U_kgDOBenDgQ
https://api.github.com/users/plamb-viso/orgs
https://api.github.com/users/plamb-viso/received_events
https://api.github.com/users/plamb-viso/repos
https://api.github.com/users/plamb-viso/starred{/owner}{/repo}
https://api.github.com/users/plamb-viso/subscriptions
User
https://api.github.com/users/plamb-viso
public
MEMBER
Did you close it because you found that it was due to the incorrect Feature types ?
2022-05-16T14:56:48Z
https://github.com/huggingface/datasets/issues/4349#issuecomment-1127780400
https://api.github.com/repos/huggingface/datasets/issues/4349
IC_kwDODunzps5DOJAw
https://api.github.com/repos/huggingface/datasets/issues/comments/1127780400/reactions
2022-05-16T14:56:48Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1127780400
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
Yeah-- my analysis of the issue was wrong in this one so I just closed it while linking to the new issue
2022-05-16T15:46:54Z
https://github.com/huggingface/datasets/issues/4349#issuecomment-1127838212
https://api.github.com/repos/huggingface/datasets/issues/4349
IC_kwDODunzps5DOXIE
https://api.github.com/repos/huggingface/datasets/issues/comments/1127838212/reactions
2022-05-16T15:46:54Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1127838212
https://avatars.githubusercontent.com/u/99206017?v=4
https://api.github.com/users/plamb-viso/events{/privacy}
https://api.github.com/users/plamb-viso/followers
https://api.github.com/users/plamb-viso/following{/other_user}
https://api.github.com/users/plamb-viso/gists{/gist_id}
https://github.com/plamb-viso
plamb-viso
U_kgDOBenDgQ
https://api.github.com/users/plamb-viso/orgs
https://api.github.com/users/plamb-viso/received_events
https://api.github.com/users/plamb-viso/repos
https://api.github.com/users/plamb-viso/starred{/owner}{/repo}
https://api.github.com/users/plamb-viso/subscriptions
User
https://api.github.com/users/plamb-viso
public
NONE
I met with the same problem when doing some experiments about layoutlm. I tried to set the writer_batch_size to 1, and the error still exists. Is there any solutions to this problem?
2022-06-02T08:39:16Z
https://github.com/huggingface/datasets/issues/4349#issuecomment-1144603496
https://api.github.com/repos/huggingface/datasets/issues/4349
IC_kwDODunzps5EOUNo
https://api.github.com/repos/huggingface/datasets/issues/comments/1144603496/reactions
2022-06-02T08:39:16Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1144603496
https://avatars.githubusercontent.com/u/7080798?v=4
https://api.github.com/users/Cppowboy/events{/privacy}
https://api.github.com/users/Cppowboy/followers
https://api.github.com/users/Cppowboy/following{/other_user}
https://api.github.com/users/Cppowboy/gists{/gist_id}
https://github.com/Cppowboy
Cppowboy
MDQ6VXNlcjcwODA3OTg=
https://api.github.com/users/Cppowboy/orgs
https://api.github.com/users/Cppowboy/received_events
https://api.github.com/users/Cppowboy/repos
https://api.github.com/users/Cppowboy/starred{/owner}{/repo}
https://api.github.com/users/Cppowboy/subscriptions
User
https://api.github.com/users/Cppowboy
public
NONE
The problem lies in how your Features are defined. It's erroring out when it actually goes to write them to disk
2022-06-02T12:51:11Z
https://github.com/huggingface/datasets/issues/4349#issuecomment-1144826979
https://api.github.com/repos/huggingface/datasets/issues/4349
IC_kwDODunzps5EPKxj
https://api.github.com/repos/huggingface/datasets/issues/comments/1144826979/reactions
2022-06-02T12:51:11Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1144826979
https://avatars.githubusercontent.com/u/99206017?v=4
https://api.github.com/users/plamb-viso/events{/privacy}
https://api.github.com/users/plamb-viso/followers
https://api.github.com/users/plamb-viso/following{/other_user}
https://api.github.com/users/plamb-viso/gists{/gist_id}
https://github.com/plamb-viso
plamb-viso
U_kgDOBenDgQ
https://api.github.com/users/plamb-viso/orgs
https://api.github.com/users/plamb-viso/received_events
https://api.github.com/users/plamb-viso/repos
https://api.github.com/users/plamb-viso/starred{/owner}{/repo}
https://api.github.com/users/plamb-viso/subscriptions
User
https://api.github.com/users/plamb-viso
public
## Describe the bug
If the the value of `writer_batch_size` is less than the total number of instances in the dataset it will fail at that same number of instances. If it is greater than the total number of instances, it fails on the last instance.

Context:
I am attempting to fine-tune a pre-trained HuggingFace transformers model called LayoutLMv2. This model takes three inputs: document images, words and word bounding boxes. [The Processor for this model has two options](https://huggingface.co/docs/transformers/model_doc/layoutlmv2#usage-layoutlmv2processor), the default is passing a document to the Processor and allowing it to create images of the document and use PyTesseract to perform OCR and generate words/bounding boxes. The other option is to provide `revision="no_ocr"` to the pre-trained model which allows you to use your own OCR results (in my case, Amazon Textract) so you have to provide the image, words and bounding boxes yourself. I am using this second option which might be good context for the bug.

I am using the Dataset.map() paradigm to create these three inputs, encode them and save the dataset. Note that my documents (data instances) on average are fairly large and can range from 1 page up to 300 pages.
Code I am using is provided below

## Steps to reproduce the bug
I do not have explicit sample code, but I will paste the code I'm using in case reading it helps. When `.map()` is called, the dataset has 2933 rows, many of which represent large pdf documents.
```python
def get_encoded_data(data):
    dataset = Dataset.from_pandas(data)
    unique_labels = data['label'].unique()
    features = Features({
        'image': Array3D(dtype="int64", shape=(3, 224, 224)),
        'input_ids': Sequence(feature=Value(dtype='int64')),
        'attention_mask': Sequence(Value(dtype='int64')),
        'token_type_ids': Sequence(Value(dtype='int64')),
        'bbox': Array2D(dtype="int64", shape=(512, 4)),
        'label': ClassLabel(num_classes=len(unique_labels), names=unique_labels),
    })

    encoded_dataset = dataset.map(preprocess_data, features=features, remove_columns=dataset.column_names, writer_batch_size=dataset.num_rows+1)
    encoded_dataset.save_to_disk(TRAINING_DATA_PATH + ENCODED_DATASET_NAME)
    encoded_dataset.set_format(type="torch")
    return encoded_dataset
```
```python
PROCESSOR = LayoutLMv2Processor.from_pretrained(MODEL_PATH, revision="no_ocr", use_fast=False)

def preprocess_data(examples):
    directory = os.path.join(FILES_PATH, examples['file_location'])
    images_dir = os.path.join(directory, PDF_IMAGE_DIR)
    textract_response_path = os.path.join(directory, 'textract.json')
    doc_meta_path = os.path.join(directory, 'doc_meta.json')
    textract_document = get_textract_document(textract_response_path, doc_meta_path)
    images, words, bboxes = get_doc_training_data(images_dir, textract_document)
    encoded_inputs = PROCESSOR(images, words, boxes=bboxes, padding="max_length", truncation=True)
    # https://github.com/NielsRogge/Transformers-Tutorials/issues/36
    encoded_inputs["image"] = np.array(encoded_inputs["image"])
    encoded_inputs["label"] = examples['label_id']
    return encoded_inputs
```

## Expected results
My expectation is that `writer_batch_size` allows one to simply trade off performance and memory requirements, not that it must be a specific number for `.map()` to function correctly.

## Actual results
If writer_batch_size is set to a value less than the number of rows, I get either:

```
OverflowError: There was an overflow with type <class 'list'>. Try to reduce writer_batch_size to have batches smaller than 2GB.
(offset overflow while concatenating arrays)
```
or simply

```
zsh: killed     python doc_classification.py

UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
```

If it is greater than the number of rows, i get the `zsh: killed` error above

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 2.1.0
- Platform: macOS-12.2.1-arm64-arm-64bit
- Python version: 3.9.12
- PyArrow version: 6.0.1
- Pandas version: 1.4.2
