Fix map() example in datasets documentation: define tokenizer before use
NONE
Hi @lhoestq, just a gentle follow-up on this doc fix PR (#7704). Let me know if any changes are needed — happy to update.
Hope this improvement helps users run the example without confusion!
2025-08-01T13:48:35Z
https://github.com/huggingface/datasets/pull/7704#issuecomment-3144658468
https://api.github.com/repos/huggingface/datasets/issues/7704
IC_kwDODunzps67b64k
https://api.github.com/repos/huggingface/datasets/issues/comments/3144658468/reactions
2025-08-01T13:48:35Z
https://api.github.com/repos/huggingface/datasets/issues/comments/3144658468
https://avatars.githubusercontent.com/u/183703408?v=4
https://api.github.com/users/Sanjaykumar030/events{/privacy}
https://api.github.com/users/Sanjaykumar030/followers
https://api.github.com/users/Sanjaykumar030/following{/other_user}
https://api.github.com/users/Sanjaykumar030/gists{/gist_id}
https://github.com/Sanjaykumar030
Sanjaykumar030
U_kgDOCvMXcA
https://api.github.com/users/Sanjaykumar030/orgs
https://api.github.com/users/Sanjaykumar030/received_events
https://api.github.com/users/Sanjaykumar030/repos
https://api.github.com/users/Sanjaykumar030/starred{/owner}{/repo}
https://api.github.com/users/Sanjaykumar030/subscriptions
User
https://api.github.com/users/Sanjaykumar030
public
## Problem
The current datasets.Dataset.map() example in the documentation demonstrates batched processing using a tokenizer object without defining or importing it. This causes a NameError when users copy and run the example as-is, breaking the expected seamless experience.

## Correction
This PR fixes the issue by explicitly importing and initializing the tokenizer using the Transformers library (AutoTokenizer.from_pretrained("bert-base-uncased")), making the example self-contained and runnable without errors.
This will help new users understand the workflow and apply the method correctly.

Closes #7703 

