Support async functions in map()
NONE
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_7384). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
2025-02-03T18:21:18Z
https://github.com/huggingface/datasets/pull/7384#issuecomment-2631740405
https://api.github.com/repos/huggingface/datasets/issues/7384
IC_kwDODunzps6c3Sv1
https://api.github.com/repos/huggingface/datasets/issues/comments/2631740405/reactions
2025-02-03T18:21:18Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2631740405
https://avatars.githubusercontent.com/u/99929124?v=4
https://api.github.com/users/HuggingFaceDocBuilderDev/events{/privacy}
https://api.github.com/users/HuggingFaceDocBuilderDev/followers
https://api.github.com/users/HuggingFaceDocBuilderDev/following{/other_user}
https://api.github.com/users/HuggingFaceDocBuilderDev/gists{/gist_id}
https://github.com/HuggingFaceDocBuilderDev
HuggingFaceDocBuilderDev
U_kgDOBfTMJA
https://api.github.com/users/HuggingFaceDocBuilderDev/orgs
https://api.github.com/users/HuggingFaceDocBuilderDev/received_events
https://api.github.com/users/HuggingFaceDocBuilderDev/repos
https://api.github.com/users/HuggingFaceDocBuilderDev/starred{/owner}{/repo}
https://api.github.com/users/HuggingFaceDocBuilderDev/subscriptions
User
https://api.github.com/users/HuggingFaceDocBuilderDev
public
MEMBER
example of what you can do with it:

```python
import aiohttp
from huggingface_hub import get_token

from datasets import Dataset


API_URL = "https://api-inference.huggingface.co/models/microsoft/Phi-3-mini-4k-instruct/v1/chat/completions"
PROMPT = "What is this text mainly about ? Here is the text:\n\n```\n{Problem}\n```\n\nReply in one or two words."

async def query(example):
    headers = {"Authorization": f"Bearer {get_token()}", "Content-Type": "application/json"}
    json = {"messages": [{"role": "user", "content": PROMPT.format(Problem=example["Problem"])}], "max_tokens": 20, "seed": 42}
    async with aiohttp.ClientSession() as session, session.post(API_URL, headers=headers, json=json) as response:
        output = await response.json()
        return {"output": output["choices"][0]["message"]["content"]}

ds = Dataset.from_dict({"Problem": ["1 + 1"] * 10})
ds = ds.map(query)
print(ds[0])
# {'Problem': '1 + 1', 'output': 'Arithmetic'}
```
2025-02-12T17:33:52Z
https://github.com/huggingface/datasets/pull/7384#issuecomment-2654411744
https://api.github.com/repos/huggingface/datasets/issues/7384
IC_kwDODunzps6eNxvg
https://api.github.com/repos/huggingface/datasets/issues/comments/2654411744/reactions
2025-02-13T14:01:13Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2654411744
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
e.g. to download images or call an inference API like HF or vLLM

```python
import asyncio
import random

from datasets import Dataset


async def f(x):
    await asyncio.sleep(random.random())

ds = Dataset.from_dict({"data": range(100)})
ds.map(f)
# Map: 100%|█████████████████████████████| 100/100 [00:01<00:00, 99.81 examples/s]
```

TODO
- [x] clean code (right now it's a big copy paste)
- [x] batched
- [x] Dataset.map()
- [x] IterableDataset.map()
- [x] Dataset.filter()
- [x] IterableDataset.filter()
- [x] test
- [x] docs