`DatasetBuilder._split_generators` incomplete type annotation
COLLABORATOR
Good catch! Feel free to open a PR with the suggested fix :).
2024-04-10T16:43:41Z
https://github.com/huggingface/datasets/issues/6798#issuecomment-2048023374
https://api.github.com/repos/huggingface/datasets/issues/6798
IC_kwDODunzps56EltO
https://api.github.com/repos/huggingface/datasets/issues/comments/2048023374/reactions
2024-04-10T16:43:41Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2048023374
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
CONTRIBUTOR
There is also the [`MockDownloadManager`](https://github.com/JonasLoos/datasets/blob/main/src/datasets/download/mock_download_manager.py#L33), which seems like it might get passed here too. However, to me, it doesn't really seem relevant to the users of the datasets library, so I would just ignore it. What do you think, @mariosasko?
2024-04-10T17:45:54Z
https://github.com/huggingface/datasets/issues/6798#issuecomment-2048133750
https://api.github.com/repos/huggingface/datasets/issues/6798
IC_kwDODunzps56FAp2
https://api.github.com/repos/huggingface/datasets/issues/comments/2048133750/reactions
2024-04-10T17:45:54Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2048133750
https://avatars.githubusercontent.com/u/33965649?v=4
https://api.github.com/users/JonasLoos/events{/privacy}
https://api.github.com/users/JonasLoos/followers
https://api.github.com/users/JonasLoos/following{/other_user}
https://api.github.com/users/JonasLoos/gists{/gist_id}
https://github.com/JonasLoos
JonasLoos
MDQ6VXNlcjMzOTY1NjQ5
https://api.github.com/users/JonasLoos/orgs
https://api.github.com/users/JonasLoos/received_events
https://api.github.com/users/JonasLoos/repos
https://api.github.com/users/JonasLoos/starred{/owner}{/repo}
https://api.github.com/users/JonasLoos/subscriptions
User
https://api.github.com/users/JonasLoos
public
COLLABORATOR
The API (`dummy_data` CLI command ) that uses the `MockDownloadManager` has been deprecated, so ignoring it sounds good!
2024-04-11T14:24:12Z
https://github.com/huggingface/datasets/issues/6798#issuecomment-2049816060
https://api.github.com/repos/huggingface/datasets/issues/6798
IC_kwDODunzps56LbX8
https://api.github.com/repos/huggingface/datasets/issues/comments/2049816060/reactions
2024-04-11T14:24:12Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2049816060
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
### Describe the bug

The [`DatasetBuilder._split_generators`](https://github.com/huggingface/datasets/blob/0f27d7b77c73412cfc50b24354bfd7a3e838202f/src/datasets/builder.py#L1449) function has currently the following signature:

```python
class DatasetBuilder:
    def _split_generators(self, dl_manager: DownloadManager):
        ...
```

However, the `dl_manager` argument can also be of type [`StreamingDownloadManager`](https://github.com/huggingface/datasets/blob/0f27d7b77c73412cfc50b24354bfd7a3e838202f/src/datasets/download/streaming_download_manager.py#L962), which has different functionality. For example, the `download` function doesn't download, but rather just returns the given url(s). 

I suggest changing the function signature to:

```python
class DatasetBuilder:
    def _split_generators(self, dl_manager: Union[DownloadManager, StreamingDownloadManager]):
        ...
```

and also adjust the docstring accordingly.

I would like to create a Pull Request to fix this, and have the following questions:
* Are there also other options than `DownloadManager`, and `StreamingDownloadManager`?
* Should this also be changed in other functions?

### Steps to reproduce the bug

Minimal example to print the different class names:

```python
import tempfile
from datasets import load_dataset

example = b'''
from datasets import GeneratorBasedBuilder, DatasetInfo, Features, Value, SplitGenerator

class Test(GeneratorBasedBuilder):
    def _info(self):
        return DatasetInfo(features=Features({"x": Value("int64")}))
    def _split_generators(self, dl_manager):
        print(type(dl_manager))
        return [SplitGenerator('test')]
    def _generate_examples(self):
        yield 0, {'x': 42}
'''

with tempfile.NamedTemporaryFile(suffix='.py') as f:
    f.write(example)
    f.flush()
    load_dataset(f.name, streaming=False)
    load_dataset(f.name, streaming=True)
```

### Expected behavior

complete type annotations

### Environment info

/