Integration with AugLy
COLLABORATOR
Hi,

you can define your own custom formatting with `Dataset.set_transform()` and then run the tokenizer with the batches of augmented data as follows:
```python
dset = load_dataset("imdb", split="train")  # Let's say we are working with the IMDB dataset
dset.set_transform(lambda ex: {"text": augly_text_augmentation(ex["text"])}, columns="text", output_all_columns=True)
dataloader = torch.utils.data.DataLoader(dset, batch_size=32)
for epoch in range(5):
    for batch in dataloader:
       tokenizer_output = tokenizer(batch.pop("text"), padding=True, truncation=True, return_tensors="pt")
       batch.update(tokenizer_output)
       output = model(**batch)
       ...
```
2021-07-11T17:08:11Z
https://github.com/huggingface/datasets/issues/2622#issuecomment-877833351
https://api.github.com/repos/huggingface/datasets/issues/2622
MDEyOklzc3VlQ29tbWVudDg3NzgzMzM1MQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/877833351/reactions
2021-07-11T17:08:11Z
https://api.github.com/repos/huggingface/datasets/issues/comments/877833351
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
COLLABORATOR
Preprocessing functions/augmentations, unless super generic, should be defined in separate libraries, so I'm closing this issue.
2023-07-20T13:18:47Z
https://github.com/huggingface/datasets/issues/2622#issuecomment-1643913302
https://api.github.com/repos/huggingface/datasets/issues/2622
IC_kwDODunzps5h_CBW
https://api.github.com/repos/huggingface/datasets/issues/comments/1643913302/reactions
2023-07-20T13:18:47Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1643913302
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
**Is your feature request related to a problem? Please describe.**
Facebook recently launched a library, [AugLy](https://github.com/facebookresearch/AugLy) , that has a unified API for augmentations for image, video and text.

It would be pretty exciting to have it hooked up to HF libraries so that we can make NLP models robust to misspellings or to punctuation, or emojis etc. Plus, with Transformers supporting more CV use cases, having augmentations support becomes crucial.

**Describe the solution you'd like**
The biggest difference between augmentations and preprocessing is that preprocessing happens only once, but you are running augmentations once per epoch. AugLy operates on text directly, so this breaks the typical workflow where we would run the tokenizer once, set format to pt tensors and be ready for the Dataloader.

**Describe alternatives you've considered**

One possible way of implementing these is to make a custom Dataset class where getitem(i) runs the augmentation and the tokenizer every time, though this would slow training down considerably given we wouldn't even run the tokenizer in batches.
