Dataset order is not deterministic with ZIP archives and `iter_files`
MEMBER
Thanks for reporting ! The issue doesn't come from shuffling, but from `beans` row order not being deterministic:

https://huggingface.co/datasets/beans/blob/main/beans.py uses `dl_manager.iter_files` on ZIP archives and the file order doesn't seen to be deterministic and changes across machines
2022-10-21T09:31:01Z
https://github.com/huggingface/datasets/issues/5145#issuecomment-1286703492
https://api.github.com/repos/huggingface/datasets/issues/5145
IC_kwDODunzps5MsYmE
https://api.github.com/repos/huggingface/datasets/issues/comments/1286703492/reactions
2022-10-21T09:31:01Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1286703492
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Thank you for noticing indeed!
2022-10-21T09:32:26Z
https://github.com/huggingface/datasets/issues/5145#issuecomment-1286705046
https://api.github.com/repos/huggingface/datasets/issues/5145
IC_kwDODunzps5MsY-W
https://api.github.com/repos/huggingface/datasets/issues/comments/1286705046/reactions
2022-10-21T09:32:26Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1286705046
https://avatars.githubusercontent.com/u/9808326?v=4
https://api.github.com/users/fxmarty/events{/privacy}
https://api.github.com/users/fxmarty/followers
https://api.github.com/users/fxmarty/following{/other_user}
https://api.github.com/users/fxmarty/gists{/gist_id}
https://github.com/fxmarty
fxmarty
MDQ6VXNlcjk4MDgzMjY=
https://api.github.com/users/fxmarty/orgs
https://api.github.com/users/fxmarty/received_events
https://api.github.com/users/fxmarty/repos
https://api.github.com/users/fxmarty/starred{/owner}{/repo}
https://api.github.com/users/fxmarty/subscriptions
User
https://api.github.com/users/fxmarty
public
MEMBER
This is still a bug, so I'd keep this one open if you don't mind ;)
2022-10-21T09:32:56Z
https://github.com/huggingface/datasets/issues/5145#issuecomment-1286705584
https://api.github.com/repos/huggingface/datasets/issues/5145
IC_kwDODunzps5MsZGw
https://api.github.com/repos/huggingface/datasets/issues/comments/1286705584/reactions
2022-10-21T09:32:56Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1286705584
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
COLLABORATOR
Besides the linked PR, to make the loading process fully deterministic, I believe we should also sort the data files [here](https://github.com/huggingface/datasets/blob/df4bdd365f2abb695f113cbf8856a925bc70901b/src/datasets/data_files.py#L276) and [here](https://github.com/huggingface/datasets/blob/df4bdd365f2abb695f113cbf8856a925bc70901b/src/datasets/data_files.py#L485) (e.g. fsspec's `LocalFileSystem.glob` relies on `os.scandir`, which yields the contents in arbitrary order). My concern is the overhead of these sorts... Maybe we could introduce a new flag to `load_dataset` similar to TFDS' [`shuffle_files`](https://www.tensorflow.org/datasets/determinism#determinism_when_reading) or sort only if the number of data files is small?
2022-10-25T16:51:10Z
https://github.com/huggingface/datasets/issues/5145#issuecomment-1290859556
https://api.github.com/repos/huggingface/datasets/issues/5145
IC_kwDODunzps5M8PQk
https://api.github.com/repos/huggingface/datasets/issues/comments/1290859556/reactions
2022-10-25T16:51:10Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1290859556
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
MEMBER
We already return the result sorted at the end of `_resolve_single_pattern_locally` and `_resolve_single_pattern_in_dataset_repository` if I'm not mistaken
2022-10-25T17:15:02Z
https://github.com/huggingface/datasets/issues/5145#issuecomment-1290885941
https://api.github.com/repos/huggingface/datasets/issues/5145
IC_kwDODunzps5M8Vs1
https://api.github.com/repos/huggingface/datasets/issues/comments/1290885941/reactions
2022-10-25T17:15:02Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1290885941
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
COLLABORATOR
@lhoestq Oh, you are right. Feel free to ignore my comment.
2022-10-26T11:51:17Z
https://github.com/huggingface/datasets/issues/5145#issuecomment-1291914980
https://api.github.com/repos/huggingface/datasets/issues/5145
IC_kwDODunzps5NAQ7k
https://api.github.com/repos/huggingface/datasets/issues/comments/1291914980/reactions
2022-10-26T11:51:17Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1291914980
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
MEMBER
I think the corresponding PR is ready to be merged :hugs: 
2022-10-26T16:53:37Z
https://github.com/huggingface/datasets/issues/5145#issuecomment-1292332336
https://api.github.com/repos/huggingface/datasets/issues/5145
IC_kwDODunzps5NB20w
https://api.github.com/repos/huggingface/datasets/issues/comments/1292332336/reactions
2022-10-26T16:53:37Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1292332336
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
CONTRIBUTOR
@albertvillanova Thanks for the fix!
2022-10-27T09:51:48Z
https://github.com/huggingface/datasets/issues/5145#issuecomment-1293274450
https://api.github.com/repos/huggingface/datasets/issues/5145
IC_kwDODunzps5NFc1S
https://api.github.com/repos/huggingface/datasets/issues/comments/1293274450/reactions
2022-10-27T09:51:48Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1293274450
https://avatars.githubusercontent.com/u/9808326?v=4
https://api.github.com/users/fxmarty/events{/privacy}
https://api.github.com/users/fxmarty/followers
https://api.github.com/users/fxmarty/following{/other_user}
https://api.github.com/users/fxmarty/gists{/gist_id}
https://github.com/fxmarty
fxmarty
MDQ6VXNlcjk4MDgzMjY=
https://api.github.com/users/fxmarty/orgs
https://api.github.com/users/fxmarty/received_events
https://api.github.com/users/fxmarty/repos
https://api.github.com/users/fxmarty/starred{/owner}{/repo}
https://api.github.com/users/fxmarty/subscriptions
User
https://api.github.com/users/fxmarty
public
### Describe the bug

For the `beans` dataset (did not try on other), the order of samples is not the same on different machines. Tested on my local laptop, github actions machine, and ec2 instance. The three yield a different order.

### Steps to reproduce the bug

In a clean docker container or conda environment with datasets==2.6.1, run

```python
from datasets import load_dataset
from pprint import pprint

data = load_dataset("beans", split="validation")

pprint(data["image_file_path"])
```

### Expected behavior

The order of the images is the same on all machines.

### Environment info

On the EC2 instance:
```
- `datasets` version: 2.6.1
- Platform: Linux-4.14.291-218.527.amzn2.x86_64-x86_64-with-glibc2.2.5
- Python version: 3.7.10
- PyArrow version: 9.0.0
- Pandas version: 1.3.5
- Numpy version: not checked
```

On my local laptop:
```
- `datasets` version: 2.6.1
- Platform: Linux-5.15.0-50-generic-x86_64-with-glibc2.35
- Python version: 3.9.12
- PyArrow version: 7.0.0
- Pandas version: 1.3.5
- Numpy version: 1.23.1
```

On github actions:
```
- `datasets` version: 2.6.1
- Platform: Linux-5.15.0-1022-azure-x86_64-with-glibc2.2.5
- Python version: 3.8.14
- PyArrow version: 9.0.0
- Pandas version: 1.5.1
- Numpy version: 1.23.4
```