`push_to_hub` overwrite argument
MEMBER
Hi ! Do you mean deleting all the files ? or erasing the repository git history before push_to_hub ?
2024-10-22T15:53:20Z
https://github.com/huggingface/datasets/issues/7241#issuecomment-2429660262
https://api.github.com/repos/huggingface/datasets/issues/7241
IC_kwDODunzps6Q0axm
https://api.github.com/repos/huggingface/datasets/issues/comments/2429660262/reactions
2024-10-22T15:53:20Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2429660262
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
Hi! I meant the latter.
2024-10-22T16:30:36Z
https://github.com/huggingface/datasets/issues/7241#issuecomment-2429742765
https://api.github.com/repos/huggingface/datasets/issues/7241
IC_kwDODunzps6Q0u6t
https://api.github.com/repos/huggingface/datasets/issues/comments/2429742765/reactions
2024-10-22T16:30:36Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2429742765
https://avatars.githubusercontent.com/u/60838378?v=4
https://api.github.com/users/ceferisbarov/events{/privacy}
https://api.github.com/users/ceferisbarov/followers
https://api.github.com/users/ceferisbarov/following{/other_user}
https://api.github.com/users/ceferisbarov/gists{/gist_id}
https://github.com/ceferisbarov
ceferisbarov
MDQ6VXNlcjYwODM4Mzc4
https://api.github.com/users/ceferisbarov/orgs
https://api.github.com/users/ceferisbarov/received_events
https://api.github.com/users/ceferisbarov/repos
https://api.github.com/users/ceferisbarov/starred{/owner}{/repo}
https://api.github.com/users/ceferisbarov/subscriptions
User
https://api.github.com/users/ceferisbarov
public
MEMBER
I don't think there is a `huggingface_hub` utility to erase the git history, cc @Wauplin maybe ?
2024-10-23T13:24:37Z
https://github.com/huggingface/datasets/issues/7241#issuecomment-2432166407
https://api.github.com/repos/huggingface/datasets/issues/7241
IC_kwDODunzps6Q9-oH
https://api.github.com/repos/huggingface/datasets/issues/comments/2432166407/reactions
2024-10-23T13:24:37Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2432166407
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
What is the goal exactly of deleting all the git history without deleting the repo? 
2024-10-23T13:41:32Z
https://github.com/huggingface/datasets/issues/7241#issuecomment-2432238360
https://api.github.com/repos/huggingface/datasets/issues/7241
IC_kwDODunzps6Q-QMY
https://api.github.com/repos/huggingface/datasets/issues/comments/2432238360/reactions
2024-10-23T13:41:32Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2432238360
https://avatars.githubusercontent.com/u/11801849?v=4
https://api.github.com/users/Wauplin/events{/privacy}
https://api.github.com/users/Wauplin/followers
https://api.github.com/users/Wauplin/following{/other_user}
https://api.github.com/users/Wauplin/gists{/gist_id}
https://github.com/Wauplin
Wauplin
MDQ6VXNlcjExODAxODQ5
https://api.github.com/users/Wauplin/orgs
https://api.github.com/users/Wauplin/received_events
https://api.github.com/users/Wauplin/repos
https://api.github.com/users/Wauplin/starred{/owner}{/repo}
https://api.github.com/users/Wauplin/subscriptions
User
https://api.github.com/users/Wauplin
public
CONTRIBUTOR
You can use [`super_squash_commit`](https://huggingface.co/docs/huggingface_hub/main/en/package_reference/hf_api#huggingface_hub.HfApi.super_squash_history) to squash all the commits into a single one, hence deleting the git history. This is not exactly what you asked for since it squashes the commits for a specific revision (example: "all commits on main"). This means that if other branches exists, they are kept the same. Also if some PRs are already opened on the repo, they will become unmergeable since the commits will have diverted.
2024-10-23T13:43:23Z
https://github.com/huggingface/datasets/issues/7241#issuecomment-2432246783
https://api.github.com/repos/huggingface/datasets/issues/7241
IC_kwDODunzps6Q-SP_
https://api.github.com/repos/huggingface/datasets/issues/comments/2432246783/reactions
2024-10-23T13:43:23Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2432246783
https://avatars.githubusercontent.com/u/11801849?v=4
https://api.github.com/users/Wauplin/events{/privacy}
https://api.github.com/users/Wauplin/followers
https://api.github.com/users/Wauplin/following{/other_user}
https://api.github.com/users/Wauplin/gists{/gist_id}
https://github.com/Wauplin
Wauplin
MDQ6VXNlcjExODAxODQ5
https://api.github.com/users/Wauplin/orgs
https://api.github.com/users/Wauplin/received_events
https://api.github.com/users/Wauplin/repos
https://api.github.com/users/Wauplin/starred{/owner}{/repo}
https://api.github.com/users/Wauplin/subscriptions
User
https://api.github.com/users/Wauplin
public
MEMBER
So the solution is:

```python
from huggingface_hub import HfApi
repo_id = "username/dataset_name"
ds.push_to_hub(repo_id)
HfApi().super_squash_commit(repo_id)
```

This way you erase previous git history to end up with only 1 commit containing your dataset.
Still, I'd be curious why it's important in your case. Is it to save storage space ? or to disallow loading old versions of the data ?
2024-10-23T17:28:05Z
https://github.com/huggingface/datasets/issues/7241#issuecomment-2432946423
https://api.github.com/repos/huggingface/datasets/issues/7241
IC_kwDODunzps6RA9D3
https://api.github.com/repos/huggingface/datasets/issues/comments/2432946423/reactions
2024-10-23T17:28:50Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2432946423
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
Thanks, everyone! I am building a new dataset and playing around with column names, splits, etc. Sometimes I push to the hub to share it with other teammates, I don't want those variations to be part of the repo. Deleting the repo from the website takes a little time, but it also loses repo settings that I have set, since I always set it to public with manually approved requests.

BTW, I had to write `HfApi().super_squash_history(repo_id, repo_type="dataset")`, but otherwise it works.
2024-10-23T20:24:17Z
https://github.com/huggingface/datasets/issues/7241#issuecomment-2433368151
https://api.github.com/repos/huggingface/datasets/issues/7241
IC_kwDODunzps6RCkBX
https://api.github.com/repos/huggingface/datasets/issues/comments/2433368151/reactions
2024-10-23T20:24:17Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2433368151
https://avatars.githubusercontent.com/u/60838378?v=4
https://api.github.com/users/ceferisbarov/events{/privacy}
https://api.github.com/users/ceferisbarov/followers
https://api.github.com/users/ceferisbarov/following{/other_user}
https://api.github.com/users/ceferisbarov/gists{/gist_id}
https://github.com/ceferisbarov
ceferisbarov
MDQ6VXNlcjYwODM4Mzc4
https://api.github.com/users/ceferisbarov/orgs
https://api.github.com/users/ceferisbarov/received_events
https://api.github.com/users/ceferisbarov/repos
https://api.github.com/users/ceferisbarov/starred{/owner}{/repo}
https://api.github.com/users/ceferisbarov/subscriptions
User
https://api.github.com/users/ceferisbarov
public
CONTRIBUTOR
@ceferisbarov just to let you know, recreating a gated repo + granting access to your teammates is something that you can automate with something like this (not fully tested but should work):

```py
from huggingface_hub import HfApi

api = HfApi()
api.delete_repo(repo_id, repo_type="dataset", missing_ok=True)
api.create_repo(repo_id, repo_type="dataset", private=False)
api.update_repo_settings(repo_id, repo_type="dataset", gated="manual")
for user in ["user1", "user2"]  # list of teammates
    api.grant_access(repo_id, user, repo_type="dataset")
```

I think it'd be a better solution than squashing commits (which is more of a hack), typically if you are using the dataset viewer.
2024-10-24T09:37:22Z
https://github.com/huggingface/datasets/issues/7241#issuecomment-2434783993
https://api.github.com/repos/huggingface/datasets/issues/7241
IC_kwDODunzps6RH9r5
https://api.github.com/repos/huggingface/datasets/issues/comments/2434783993/reactions
2024-10-24T09:37:22Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2434783993
https://avatars.githubusercontent.com/u/11801849?v=4
https://api.github.com/users/Wauplin/events{/privacy}
https://api.github.com/users/Wauplin/followers
https://api.github.com/users/Wauplin/following{/other_user}
https://api.github.com/users/Wauplin/gists{/gist_id}
https://github.com/Wauplin
Wauplin
MDQ6VXNlcjExODAxODQ5
https://api.github.com/users/Wauplin/orgs
https://api.github.com/users/Wauplin/received_events
https://api.github.com/users/Wauplin/repos
https://api.github.com/users/Wauplin/starred{/owner}{/repo}
https://api.github.com/users/Wauplin/subscriptions
User
https://api.github.com/users/Wauplin
public
NONE
This is great, @Wauplin. If we can achieve this with HfApi, then we probably don't need to add another parameter to push_to_hub. I am closing the issue.
2024-10-24T17:39:08Z
https://github.com/huggingface/datasets/issues/7241#issuecomment-2435943603
https://api.github.com/repos/huggingface/datasets/issues/7241
IC_kwDODunzps6RMYyz
https://api.github.com/repos/huggingface/datasets/issues/comments/2435943603/reactions
2024-10-24T17:39:08Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2435943603
https://avatars.githubusercontent.com/u/60838378?v=4
https://api.github.com/users/ceferisbarov/events{/privacy}
https://api.github.com/users/ceferisbarov/followers
https://api.github.com/users/ceferisbarov/following{/other_user}
https://api.github.com/users/ceferisbarov/gists{/gist_id}
https://github.com/ceferisbarov
ceferisbarov
MDQ6VXNlcjYwODM4Mzc4
https://api.github.com/users/ceferisbarov/orgs
https://api.github.com/users/ceferisbarov/received_events
https://api.github.com/users/ceferisbarov/repos
https://api.github.com/users/ceferisbarov/starred{/owner}{/repo}
https://api.github.com/users/ceferisbarov/subscriptions
User
https://api.github.com/users/ceferisbarov
public
### Feature request

Add an `overwrite` argument to the `push_to_hub` method.

### Motivation

I want to overwrite a repo without deleting it on Hugging Face. Is this possible? I couldn't find anything in the documentation or tutorials.

### Your contribution

I can create a PR.