Dataset sharding non-contiguous?
CONTRIBUTOR
This was silly; I was sure I'd looked for a `contiguous` argument, and was certain there wasn't one the first time I looked :smile:

Sorry about that.
2022-06-26T14:36:20Z
https://github.com/huggingface/datasets/issues/4570#issuecomment-1166551668
https://api.github.com/repos/huggingface/datasets/issues/4570
IC_kwDODunzps5FiCp0
https://api.github.com/repos/huggingface/datasets/issues/comments/1166551668/reactions
2022-06-26T14:36:20Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1166551668
https://avatars.githubusercontent.com/u/3664563?v=4
https://api.github.com/users/cakiki/events{/privacy}
https://api.github.com/users/cakiki/followers
https://api.github.com/users/cakiki/following{/other_user}
https://api.github.com/users/cakiki/gists{/gist_id}
https://github.com/cakiki
cakiki
MDQ6VXNlcjM2NjQ1NjM=
https://api.github.com/users/cakiki/orgs
https://api.github.com/users/cakiki/received_events
https://api.github.com/users/cakiki/repos
https://api.github.com/users/cakiki/starred{/owner}{/repo}
https://api.github.com/users/cakiki/subscriptions
User
https://api.github.com/users/cakiki
public
COLLABORATOR
Hi! You can pass `contiguous=True` to `.shard()` get contiguous shards. More info on this and the default behavior can be found in the [docs](https://huggingface.co/docs/datasets/v2.3.2/en/package_reference/main_classes#datasets.Dataset.shard).

EDIT: Answered as you closed the thread 😄 
2022-06-26T14:37:21Z
https://github.com/huggingface/datasets/issues/4570#issuecomment-1166551921
https://api.github.com/repos/huggingface/datasets/issues/4570
IC_kwDODunzps5FiCtx
https://api.github.com/repos/huggingface/datasets/issues/comments/1166551921/reactions
2022-06-26T14:38:05Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1166551921
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
CONTRIBUTOR
Hahaha I'm sorry; my excuse is: it's Sunday. (Which makes me all the more grateful for your response :smiley: 
2022-06-26T14:39:52Z
https://github.com/huggingface/datasets/issues/4570#issuecomment-1166552499
https://api.github.com/repos/huggingface/datasets/issues/4570
IC_kwDODunzps5FiC2z
https://api.github.com/repos/huggingface/datasets/issues/comments/1166552499/reactions
2022-06-26T14:39:52Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1166552499
https://avatars.githubusercontent.com/u/3664563?v=4
https://api.github.com/users/cakiki/events{/privacy}
https://api.github.com/users/cakiki/followers
https://api.github.com/users/cakiki/following{/other_user}
https://api.github.com/users/cakiki/gists{/gist_id}
https://github.com/cakiki
cakiki
MDQ6VXNlcjM2NjQ1NjM=
https://api.github.com/users/cakiki/orgs
https://api.github.com/users/cakiki/received_events
https://api.github.com/users/cakiki/repos
https://api.github.com/users/cakiki/starred{/owner}{/repo}
https://api.github.com/users/cakiki/subscriptions
User
https://api.github.com/users/cakiki
public
CONTRIBUTOR
@mariosasko Sorry for reviving this, but I was curious as to why `contiguous=False` was the default. This might be a personal bias, but I feel that a user would expect the opposite to be the default. :thinking: 
2022-06-28T15:51:47Z
https://github.com/huggingface/datasets/issues/4570#issuecomment-1168899461
https://api.github.com/repos/huggingface/datasets/issues/4570
IC_kwDODunzps5Fq_2F
https://api.github.com/repos/huggingface/datasets/issues/comments/1168899461/reactions
2022-06-28T15:51:47Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1168899461
https://avatars.githubusercontent.com/u/3664563?v=4
https://api.github.com/users/cakiki/events{/privacy}
https://api.github.com/users/cakiki/followers
https://api.github.com/users/cakiki/following{/other_user}
https://api.github.com/users/cakiki/gists{/gist_id}
https://github.com/cakiki
cakiki
MDQ6VXNlcjM2NjQ1NjM=
https://api.github.com/users/cakiki/orgs
https://api.github.com/users/cakiki/received_events
https://api.github.com/users/cakiki/repos
https://api.github.com/users/cakiki/starred{/owner}{/repo}
https://api.github.com/users/cakiki/subscriptions
User
https://api.github.com/users/cakiki
public
COLLABORATOR
This project started as a fork of TFDS, and `contiguous=False` is the default behavior [there](https://www.tensorflow.org/api_docs/python/tf/data/Dataset#shard).
2022-06-30T11:00:47Z
https://github.com/huggingface/datasets/issues/4570#issuecomment-1171077552
https://api.github.com/repos/huggingface/datasets/issues/4570
IC_kwDODunzps5FzTmw
https://api.github.com/repos/huggingface/datasets/issues/comments/1171077552/reactions
2022-06-30T11:00:47Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1171077552
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
## Describe the bug
I'm not sure if this is a bug; more likely normal behavior but i wanted to double check.
Is it normal that `datasets.shard` does not produce chunks that, when concatenated produce the original ordering of the sharded dataset? 

This might be related to this pull request (https://github.com/huggingface/datasets/pull/4466) but I have to admit I did not properly look into the changes made.

## Steps to reproduce the bug
```python
max_shard_size = convert_file_size_to_int('300MB')
dataset_nbytes = dataset.data.nbytes
num_shards = int(dataset_nbytes / max_shard_size) + 1
num_shards = max(num_shards, 1)
print(f"{num_shards=}")
for shard_index in range(num_shards):
    shard = dataset.shard(num_shards=num_shards, index=shard_index)
    shard.to_parquet(f"tokenized/tokenized-{shard_index:03d}.parquet")
os.listdir('tokenized/')
```

## Expected results
I expected the shards to match the order of the data of the original dataset; i.e. `dataset[10]` being the same as `shard_1[10]` for example

## Actual results
Only the first element is the same; i.e. `dataset[0]` is the same as `shard_1[0]`

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 2.3.2
- Platform: Linux-4.15.0-176-generic-x86_64-with-glibc2.31
- Python version: 3.10.4
- PyArrow version: 8.0.0
- Pandas version: 1.4.2

