Consider using "Sequence" instead of "List"
COLLABORATOR
Hi! Linking a comment to provide more info on the issue: https://stackoverflow.com/a/39458225. This means we should replace all (most of) the occurrences of `List` with `Sequence` in function signatures.

@tranhd95 Would you be interested in submitting a PR?
2022-12-13T17:18:51Z
https://github.com/huggingface/datasets/issues/5354#issuecomment-1349133312
https://api.github.com/repos/huggingface/datasets/issues/5354
IC_kwDODunzps5QaiQA
https://api.github.com/repos/huggingface/datasets/issues/comments/1349133312/reactions
2022-12-13T17:18:51Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1349133312
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
NONE
Hi all! I tried to reproduce this issue and didn't work for me. Also in your example i noticed that the variables have different names: `list_of_filenames` and `list_of_files`, could this be related to that?
```python
#I found random data in parquet format:
!wget "https://github.com/Teradata/kylo/raw/master/samples/sample-data/parquet/userdata1.parquet"
!wget "https://github.com/Teradata/kylo/raw/master/samples/sample-data/parquet/userdata2.parquet"

#Then i try reproduce
list_of_files = ["userdata1.parquet", "userdata2.parquet"]
ds = Dataset.from_parquet(list_of_files)
```
**My output:**
```python
WARNING:datasets.builder:Using custom data configuration default-e287d097dc54e046
Downloading and preparing dataset parquet/default to /root/.cache/huggingface/datasets/parquet/default-e287d097dc54e046/0.0.0/2a3b91fbd88a2c90d1dbbb32b460cf621d31bd5b05b934492fdef7d8d6f236ec...
Downloading data files: 100%
1/1 [00:00<00:00, 40.38it/s]
Extracting data files: 100%
1/1 [00:00<00:00, 23.43it/s]
Dataset parquet downloaded and prepared to /root/.cache/huggingface/datasets/parquet/default-e287d097dc54e046/0.0.0/2a3b91fbd88a2c90d1dbbb32b460cf621d31bd5b05b934492fdef7d8d6f236ec. Subsequent calls will reuse this data.
```
P.S. This is my first experience with open source. So do not judge strictly if I do not understand something)
2022-12-16T19:14:28Z
https://github.com/huggingface/datasets/issues/5354#issuecomment-1355455510
https://api.github.com/repos/huggingface/datasets/issues/5354
IC_kwDODunzps5QypwW
https://api.github.com/repos/huggingface/datasets/issues/comments/1355455510/reactions
2022-12-16T19:14:28Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1355455510
https://avatars.githubusercontent.com/u/86718504?v=4
https://api.github.com/users/dantema/events{/privacy}
https://api.github.com/users/dantema/followers
https://api.github.com/users/dantema/following{/other_user}
https://api.github.com/users/dantema/gists{/gist_id}
https://github.com/dantema
dantema
MDQ6VXNlcjg2NzE4NTA0
https://api.github.com/users/dantema/orgs
https://api.github.com/users/dantema/received_events
https://api.github.com/users/dantema/repos
https://api.github.com/users/dantema/starred{/owner}{/repo}
https://api.github.com/users/dantema/subscriptions
User
https://api.github.com/users/dantema
public
NONE
@dantema There is indeed a typo in variable names. Nevertheless, I'm sorry if I was not clear but the output is from `mypy` type checker. You can run the code snippet without issues. The problem is with the type checking.
2022-12-16T20:46:19Z
https://github.com/huggingface/datasets/issues/5354#issuecomment-1355585732
https://api.github.com/repos/huggingface/datasets/issues/5354
IC_kwDODunzps5QzJjE
https://api.github.com/repos/huggingface/datasets/issues/comments/1355585732/reactions
2022-12-16T20:46:19Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1355585732
https://avatars.githubusercontent.com/u/15568078?v=4
https://api.github.com/users/tranhd95/events{/privacy}
https://api.github.com/users/tranhd95/followers
https://api.github.com/users/tranhd95/following{/other_user}
https://api.github.com/users/tranhd95/gists{/gist_id}
https://github.com/tranhd95
tranhd95
MDQ6VXNlcjE1NTY4MDc4
https://api.github.com/users/tranhd95/orgs
https://api.github.com/users/tranhd95/received_events
https://api.github.com/users/tranhd95/repos
https://api.github.com/users/tranhd95/starred{/owner}{/repo}
https://api.github.com/users/tranhd95/subscriptions
User
https://api.github.com/users/tranhd95
public
NONE
However, I found out that the type annotation is actually misleading. The [`from_parquet`](https://github.com/huggingface/datasets/blob/5ef1ab1cc06c2b7a574bf2df454cd9fcb071ccb2/src/datasets/arrow_dataset.py#L1039) method should also accept list of [`PathLike`](https://github.com/huggingface/datasets/blob/main/src/datasets/utils/typing.py#L8) objects which includes [`os.PathLike`](https://docs.python.org/3/library/os.html#os.PathLike). But if I would ran the code snippet below, an exception is thrown.

**Code**
```py
from pathlib import Path

list_of_filenames = [Path("foo.parquet"), Path("bar.parquet")]
ds = Dataset.from_parquet(list_of_filenames)
```
**Output**
```py
[/usr/local/lib/python3.8/dist-packages/datasets/arrow_dataset.py](https://localhost:8080/#) in from_parquet(path_or_paths, split, features, cache_dir, keep_in_memory, columns, **kwargs)
   1071         from .io.parquet import ParquetDatasetReader
   1072 
-> 1073         return ParquetDatasetReader(
   1074             path_or_paths,
   1075             split=split,

[/usr/local/lib/python3.8/dist-packages/datasets/io/parquet.py](https://localhost:8080/#) in __init__(self, path_or_paths, split, features, cache_dir, keep_in_memory, streaming, **kwargs)
     35         path_or_paths = path_or_paths if isinstance(path_or_paths, dict) else {self.split: path_or_paths}
     36         hash = _PACKAGED_DATASETS_MODULES["parquet"][1]
---> 37         self.builder = Parquet(
     38             cache_dir=cache_dir,
     39             data_files=path_or_paths,

[/usr/local/lib/python3.8/dist-packages/datasets/builder.py](https://localhost:8080/#) in __init__(self, cache_dir, config_name, hash, base_path, info, features, use_auth_token, repo_id, data_files, data_dir, name, **config_kwargs)
    298 
    299         if data_files is not None and not isinstance(data_files, DataFilesDict):
--> 300             data_files = DataFilesDict.from_local_or_remote(
    301                 sanitize_patterns(data_files), base_path=base_path, use_auth_token=use_auth_token
    302             )

[/usr/local/lib/python3.8/dist-packages/datasets/data_files.py](https://localhost:8080/#) in from_local_or_remote(cls, patterns, base_path, allowed_extensions, use_auth_token)
    794         for key, patterns_for_key in patterns.items():
    795             out[key] = (
--> 796                 DataFilesList.from_local_or_remote(
    797                     patterns_for_key,
    798                     base_path=base_path,

[/usr/local/lib/python3.8/dist-packages/datasets/data_files.py](https://localhost:8080/#) in from_local_or_remote(cls, patterns, base_path, allowed_extensions, use_auth_token)
    762     ) -> "DataFilesList":
    763         base_path = base_path if base_path is not None else str(Path().resolve())
--> 764         data_files = resolve_patterns_locally_or_by_urls(base_path, patterns, allowed_extensions)
    765         origin_metadata = _get_origin_metadata_locally_or_by_urls(data_files, use_auth_token=use_auth_token)
    766         return cls(data_files, origin_metadata)

[/usr/local/lib/python3.8/dist-packages/datasets/data_files.py](https://localhost:8080/#) in resolve_patterns_locally_or_by_urls(base_path, patterns, allowed_extensions)
    357     data_files = []
    358     for pattern in patterns:
--> 359         if is_remote_url(pattern):
    360             data_files.append(Url(pattern))
    361         else:

[/usr/local/lib/python3.8/dist-packages/datasets/utils/file_utils.py](https://localhost:8080/#) in is_remote_url(url_or_filename)
     62 
     63 def is_remote_url(url_or_filename: str) -> bool:
---> 64     parsed = urlparse(url_or_filename)
     65     return parsed.scheme in ("http", "https", "s3", "gs", "hdfs", "ftp")
     66 

[/usr/lib/python3.8/urllib/parse.py](https://localhost:8080/#) in urlparse(url, scheme, allow_fragments)
    373     Note that we don't break the components up in smaller bits
    374     (e.g. netloc is a single string) and we don't expand % escapes."""
--> 375     url, scheme, _coerce_result = _coerce_args(url, scheme)
    376     splitresult = urlsplit(url, scheme, allow_fragments)
    377     scheme, netloc, url, query, fragment = splitresult

[/usr/lib/python3.8/urllib/parse.py](https://localhost:8080/#) in _coerce_args(*args)
    125     if str_input:
    126         return args + (_noop,)
--> 127     return _decode_args(args) + (_encode_result,)
    128 
    129 # Result objects are more helpful than simple tuples

[/usr/lib/python3.8/urllib/parse.py](https://localhost:8080/#) in _decode_args(args, encoding, errors)
    109 def _decode_args(args, encoding=_implicit_encoding,
    110                        errors=_implicit_errors):
--> 111     return tuple(x.decode(encoding, errors) if x else '' for x in args)
    112 
    113 def _coerce_args(*args):

[/usr/lib/python3.8/urllib/parse.py](https://localhost:8080/#) in <genexpr>(.0)
    109 def _decode_args(args, encoding=_implicit_encoding,
    110                        errors=_implicit_errors):
--> 111     return tuple(x.decode(encoding, errors) if x else '' for x in args)
    112 
    113 def _coerce_args(*args):

AttributeError: 'PosixPath' object has no attribute 'decode'
```

@mariosasko Should I create a new issue? 
2022-12-16T21:02:43Z
https://github.com/huggingface/datasets/issues/5354#issuecomment-1355603677
https://api.github.com/repos/huggingface/datasets/issues/5354
IC_kwDODunzps5QzN7d
https://api.github.com/repos/huggingface/datasets/issues/comments/1355603677/reactions
2022-12-16T21:02:43Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1355603677
https://avatars.githubusercontent.com/u/15568078?v=4
https://api.github.com/users/tranhd95/events{/privacy}
https://api.github.com/users/tranhd95/followers
https://api.github.com/users/tranhd95/following{/other_user}
https://api.github.com/users/tranhd95/gists{/gist_id}
https://github.com/tranhd95
tranhd95
MDQ6VXNlcjE1NTY4MDc4
https://api.github.com/users/tranhd95/orgs
https://api.github.com/users/tranhd95/received_events
https://api.github.com/users/tranhd95/repos
https://api.github.com/users/tranhd95/starred{/owner}{/repo}
https://api.github.com/users/tranhd95/subscriptions
User
https://api.github.com/users/tranhd95
public
CONTRIBUTOR
@mariosasko I would like to take this issue up. 
2023-02-01T05:14:14Z
https://github.com/huggingface/datasets/issues/5354#issuecomment-1411480359
https://api.github.com/repos/huggingface/datasets/issues/5354
IC_kwDODunzps5UIXsn
https://api.github.com/repos/huggingface/datasets/issues/comments/1411480359/reactions
2023-02-01T05:14:14Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1411480359
https://avatars.githubusercontent.com/u/22453634?v=4
https://api.github.com/users/avinashsai/events{/privacy}
https://api.github.com/users/avinashsai/followers
https://api.github.com/users/avinashsai/following{/other_user}
https://api.github.com/users/avinashsai/gists{/gist_id}
https://github.com/avinashsai
avinashsai
MDQ6VXNlcjIyNDUzNjM0
https://api.github.com/users/avinashsai/orgs
https://api.github.com/users/avinashsai/received_events
https://api.github.com/users/avinashsai/repos
https://api.github.com/users/avinashsai/starred{/owner}{/repo}
https://api.github.com/users/avinashsai/subscriptions
User
https://api.github.com/users/avinashsai
public
COLLABORATOR
@avinashsai Hi, I've assigned you the issue.

@tranhd95 Yes, feel free to report this in a new issue.
2023-02-01T13:55:13Z
https://github.com/huggingface/datasets/issues/5354#issuecomment-1412097953
https://api.github.com/repos/huggingface/datasets/issues/5354
IC_kwDODunzps5UKueh
https://api.github.com/repos/huggingface/datasets/issues/comments/1412097953/reactions
2023-02-01T13:55:13Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1412097953
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
CONTRIBUTOR
@avinashsai Are you still working on this? If not I would like to give it a try.
2023-07-26T16:25:50Z
https://github.com/huggingface/datasets/issues/5354#issuecomment-1652140242
https://api.github.com/repos/huggingface/datasets/issues/5354
IC_kwDODunzps5ieajS
https://api.github.com/repos/huggingface/datasets/issues/comments/1652140242/reactions
2023-07-26T16:25:50Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1652140242
https://avatars.githubusercontent.com/u/51715864?v=4
https://api.github.com/users/Unknown3141592/events{/privacy}
https://api.github.com/users/Unknown3141592/followers
https://api.github.com/users/Unknown3141592/following{/other_user}
https://api.github.com/users/Unknown3141592/gists{/gist_id}
https://github.com/Unknown3141592
Unknown3141592
MDQ6VXNlcjUxNzE1ODY0
https://api.github.com/users/Unknown3141592/orgs
https://api.github.com/users/Unknown3141592/received_events
https://api.github.com/users/Unknown3141592/repos
https://api.github.com/users/Unknown3141592/starred{/owner}{/repo}
https://api.github.com/users/Unknown3141592/subscriptions
User
https://api.github.com/users/Unknown3141592
public
NONE
@mariosasko I would like to take this issue up!
2023-11-20T06:21:28Z
https://github.com/huggingface/datasets/issues/5354#issuecomment-1818296050
https://api.github.com/repos/huggingface/datasets/issues/5354
IC_kwDODunzps5sYP7y
https://api.github.com/repos/huggingface/datasets/issues/comments/1818296050/reactions
2023-11-20T06:21:28Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1818296050
https://avatars.githubusercontent.com/u/85500670?v=4
https://api.github.com/users/UNCANNY69/events{/privacy}
https://api.github.com/users/UNCANNY69/followers
https://api.github.com/users/UNCANNY69/following{/other_user}
https://api.github.com/users/UNCANNY69/gists{/gist_id}
https://github.com/UNCANNY69
UNCANNY69
MDQ6VXNlcjg1NTAwNjcw
https://api.github.com/users/UNCANNY69/orgs
https://api.github.com/users/UNCANNY69/received_events
https://api.github.com/users/UNCANNY69/repos
https://api.github.com/users/UNCANNY69/starred{/owner}{/repo}
https://api.github.com/users/UNCANNY69/subscriptions
User
https://api.github.com/users/UNCANNY69
public
NONE
Hi @tranhd95 @mariosasko ,I hope you all are doing well.

I am interested in this issue, is this still open and unresolved ?

Thanks and Regards
2024-01-20T19:57:16Z
https://github.com/huggingface/datasets/issues/5354#issuecomment-1902251849
https://api.github.com/repos/huggingface/datasets/issues/5354
IC_kwDODunzps5xYg9J
https://api.github.com/repos/huggingface/datasets/issues/comments/1902251849/reactions
2024-01-20T19:57:16Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1902251849
https://avatars.githubusercontent.com/u/79078267?v=4
https://api.github.com/users/ojuschugh1/events{/privacy}
https://api.github.com/users/ojuschugh1/followers
https://api.github.com/users/ojuschugh1/following{/other_user}
https://api.github.com/users/ojuschugh1/gists{/gist_id}
https://github.com/ojuschugh1
ojuschugh1
MDQ6VXNlcjc5MDc4MjY3
https://api.github.com/users/ojuschugh1/orgs
https://api.github.com/users/ojuschugh1/received_events
https://api.github.com/users/ojuschugh1/repos
https://api.github.com/users/ojuschugh1/starred{/owner}{/repo}
https://api.github.com/users/ojuschugh1/subscriptions
User
https://api.github.com/users/ojuschugh1
public
NONE
@mariosasko I would like to take this issue up.
2024-10-05T14:38:43Z
https://github.com/huggingface/datasets/issues/5354#issuecomment-2395079394
https://api.github.com/repos/huggingface/datasets/issues/5354
IC_kwDODunzps6OwgLi
https://api.github.com/repos/huggingface/datasets/issues/comments/2395079394/reactions
2024-10-05T14:38:43Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2395079394
https://avatars.githubusercontent.com/u/141608197?v=4
https://api.github.com/users/hritikranjan1/events{/privacy}
https://api.github.com/users/hritikranjan1/followers
https://api.github.com/users/hritikranjan1/following{/other_user}
https://api.github.com/users/hritikranjan1/gists{/gist_id}
https://github.com/hritikranjan1
hritikranjan1
U_kgDOCHDFBQ
https://api.github.com/users/hritikranjan1/orgs
https://api.github.com/users/hritikranjan1/received_events
https://api.github.com/users/hritikranjan1/repos
https://api.github.com/users/hritikranjan1/starred{/owner}{/repo}
https://api.github.com/users/hritikranjan1/subscriptions
User
https://api.github.com/users/hritikranjan1
public
CONTRIBUTOR
Hi @mariosasko, I’d like to work on this issue. I’ll start by replacing List with Sequence in relevant function signatures and ensure type checking passes. Let me know if there are any specific files to prioritize.
2025-06-21T13:56:58Z
https://github.com/huggingface/datasets/issues/5354#issuecomment-2993596786
https://api.github.com/repos/huggingface/datasets/issues/5354
IC_kwDODunzps6ybqly
https://api.github.com/repos/huggingface/datasets/issues/comments/2993596786/reactions
2025-06-21T13:56:58Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2993596786
https://avatars.githubusercontent.com/u/142811259?v=4
https://api.github.com/users/ArjunJagdale/events{/privacy}
https://api.github.com/users/ArjunJagdale/followers
https://api.github.com/users/ArjunJagdale/following{/other_user}
https://api.github.com/users/ArjunJagdale/gists{/gist_id}
https://github.com/ArjunJagdale
ArjunJagdale
U_kgDOCIMgew
https://api.github.com/users/ArjunJagdale/orgs
https://api.github.com/users/ArjunJagdale/received_events
https://api.github.com/users/ArjunJagdale/repos
https://api.github.com/users/ArjunJagdale/starred{/owner}{/repo}
https://api.github.com/users/ArjunJagdale/subscriptions
User
https://api.github.com/users/ArjunJagdale
public
### Feature request

Hi, please consider using `Sequence` type annotation instead of `List` in function arguments such as in [`Dataset.from_parquet()`](https://github.com/huggingface/datasets/blob/main/src/datasets/arrow_dataset.py#L1088). It leads to type checking errors, see below.


**How to reproduce**
```py
list_of_filenames = ["foo.parquet", "bar.parquet"]
ds = Dataset.from_parquet(list_of_filenames)
```

**Expected mypy output:**
```
Success: no issues found
```

**Actual mypy output:**
```py
test.py:19: error: Argument 1 to "from_parquet" of "Dataset" has incompatible type "List[str]"; expected "Union[Union[str, bytes, PathLike[Any]], List[Union[str, bytes, PathLike[Any]]]]"  [arg-type]
test.py:19: note: "List" is invariant -- see https://mypy.readthedocs.io/en/stable/common_issues.html#variance
test.py:19: note: Consider using "Sequence" instead, which is covariant
```

**Env:** mypy 0.991, Python 3.10.0, datasets 2.7.1