Push samples to dataset on hub without having the dataset locally
MEMBER
Hi ! For now I would recommend creating a new Parquet file using `dataset_new.to_parquet()` and upload it to HF using `huggingface_hub` every time you get a new batch of data. You can name the Parquet files `0000.parquet`, `0001.parquet`, etc.

Though maybe make sure to not upload one file per sample since that would be inefficient. You can buffer your data and upload when you have enough new samples for example
2024-03-01T19:34:26Z
https://github.com/huggingface/datasets/issues/6702#issuecomment-1973798670
https://api.github.com/repos/huggingface/datasets/issues/6702
IC_kwDODunzps51pccO
https://api.github.com/repos/huggingface/datasets/issues/comments/1973798670/reactions
2024-03-01T19:34:26Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1973798670
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
This is excellent, thanks!
2024-03-08T21:08:38Z
https://github.com/huggingface/datasets/issues/6702#issuecomment-1986422484
https://api.github.com/repos/huggingface/datasets/issues/6702
IC_kwDODunzps52ZmbU
https://api.github.com/repos/huggingface/datasets/issues/comments/1986422484/reactions
2024-03-08T21:08:38Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1986422484
https://avatars.githubusercontent.com/u/17854096?v=4
https://api.github.com/users/jbdel/events{/privacy}
https://api.github.com/users/jbdel/followers
https://api.github.com/users/jbdel/following{/other_user}
https://api.github.com/users/jbdel/gists{/gist_id}
https://github.com/jbdel
jbdel
MDQ6VXNlcjE3ODU0MDk2
https://api.github.com/users/jbdel/orgs
https://api.github.com/users/jbdel/received_events
https://api.github.com/users/jbdel/repos
https://api.github.com/users/jbdel/starred{/owner}{/repo}
https://api.github.com/users/jbdel/subscriptions
User
https://api.github.com/users/jbdel
public
### Feature request

Say I have the following code:

```
from datasets import Dataset
import pandas as pd
new_data = {
    "column_1": ["value1", "value2"],
    "column_2": ["value3", "value4"],
}
df_new = pd.DataFrame(new_data)
dataset_new = Dataset.from_pandas(df_new)
# add these samples to a remote dataset
```
It would be great to have a way to push dataset_new to a remote dataset that respects the same schema. This way one would not have to do the following:
```
from datasets import load_dataset
dataset = load_dataset('username/dataset_name', use_auth_token='your_hf_token_here')
updated_dataset = dataset['train'].concatenate(dataset_new)
updated_dataset.push_to_hub('username/dataset_name', use_auth_token='your_hf_token_here')
```





### Motivation

No need to download the dataset.

### Your contribution

Maybe this feature already exists, didnt see it though. I do not have the expertise to do this.