Pushing dataset to hub crash
MEMBER
Hi ! It's been fixed by https://github.com/huggingface/datasets/pull/5598. We're doing a new release tomorrow with the fix and you'll be able to push your 100k images ;)

Basically `push_to_hub` used to fail if the remote repository already exists and has a README.md without dataset_info in the YAML tags.

In the meantime you can install datasets from source
2023-03-27T13:08:40Z
https://github.com/huggingface/datasets/issues/5672#issuecomment-1485072932
https://api.github.com/repos/huggingface/datasets/issues/5672
IC_kwDODunzps5YhGok
https://api.github.com/repos/huggingface/datasets/issues/comments/1485072932/reactions
2023-03-27T13:08:40Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1485072932
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
Hi @lhoestq ,

What version of datasets library fix this case? I am using the last `v2.10.1` and I get the same error.
2023-03-28T23:41:03Z
https://github.com/huggingface/datasets/issues/5672#issuecomment-1487745381
https://api.github.com/repos/huggingface/datasets/issues/5672
IC_kwDODunzps5YrTFl
https://api.github.com/repos/huggingface/datasets/issues/comments/1487745381/reactions
2023-03-28T23:41:03Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1487745381
https://avatars.githubusercontent.com/u/4375209?v=4
https://api.github.com/users/bukosabino/events{/privacy}
https://api.github.com/users/bukosabino/followers
https://api.github.com/users/bukosabino/following{/other_user}
https://api.github.com/users/bukosabino/gists{/gist_id}
https://github.com/bukosabino
bukosabino
MDQ6VXNlcjQzNzUyMDk=
https://api.github.com/users/bukosabino/orgs
https://api.github.com/users/bukosabino/received_events
https://api.github.com/users/bukosabino/repos
https://api.github.com/users/bukosabino/starred{/owner}{/repo}
https://api.github.com/users/bukosabino/subscriptions
User
https://api.github.com/users/bukosabino
public
MEMBER
We just released 2.11 which includes a fix :)
2023-03-29T18:36:55Z
https://github.com/huggingface/datasets/issues/5672#issuecomment-1489110924
https://api.github.com/repos/huggingface/datasets/issues/5672
IC_kwDODunzps5YwgeM
https://api.github.com/repos/huggingface/datasets/issues/comments/1489110924/reactions
2023-03-29T18:36:55Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1489110924
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
### Describe the bug

Uploading a dataset with `push_to_hub()` fails without error description. 

### Steps to reproduce the bug

Hey there,

I've built a image dataset of 100k images + text pair as described here https://huggingface.co/docs/datasets/image_dataset#imagefolder

Now I'm trying to push it to the hub but I'm running into issues. First, I tried doing it via git directly, I added all the files in git lfs and pushed but I got hit with an error saying huggingface only accept up to 10k files in a folder.

So I'm now trying with the `push_to_hub()` func as follow:

```python
from datasets import load_dataset
import os

dataset = load_dataset("imagefolder", data_dir="./data", split="train")
dataset.push_to_hub("tzvc/organization-logos", token=os.environ.get('HF_TOKEN'))
```

But again, this produces an error:

```
Resolving data files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████| 100212/100212 [00:00<00:00, 439108.61it/s]
Downloading and preparing dataset imagefolder/default to /home/contact_theochampion/.cache/huggingface/datasets/imagefolder/default-20567ffc703aa314/0.0.0/37fbb85cc714a338bea574ac6c7d0b5be5aff46c1862c1989b20e0771199e93f...
Downloading data files: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████| 100211/100211 [00:00<00:00, 149323.73it/s]
Downloading data files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 15947.92it/s]
Extracting data files: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 2245.34it/s]
Dataset imagefolder downloaded and prepared to /home/contact_theochampion/.cache/huggingface/datasets/imagefolder/default-20567ffc703aa314/0.0.0/37fbb85cc714a338bea574ac6c7d0b5be5aff46c1862c1989b20e0771199e93f. Subsequent calls will reuse this data.
Resuming upload of the dataset shards.                                                                                                                                        
Pushing dataset shards to the dataset hub: 100%|██████████████████████████████████████████████████████████████████████████████████████████████| 14/14 [00:31<00:00,  2.24s/it]
Downloading metadata: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 118/118 [00:00<00:00, 225kB/s]
Traceback (most recent call last):
  File "/home/contact_theochampion/organization-logos/push_to_hub.py", line 5, in <module>
    dataset.push_to_hub("tzvc/organization-logos", token=os.environ.get('HF_TOKEN'))
  File "/home/contact_theochampion/.local/lib/python3.9/site-packages/datasets/arrow_dataset.py", line 5245, in push_to_hub
    repo_info = dataset_infos[next(iter(dataset_infos))]
StopIteration
```

What could be happening here ?

### Expected behavior

The dataset is pushed to the hub

### Environment info

- `datasets` version: 2.10.1
- Platform: Linux-5.10.0-21-cloud-amd64-x86_64-with-glibc2.31
- Python version: 3.9.2
- PyArrow version: 11.0.0
- Pandas version: 1.5.3