how can I combine 2 dataset with different/same features?
MEMBER
Hi ! Currently we don't have a way to `zip` datasets but we plan to add this soon :)
For now you'll need to use `map` to add the fields from one dataset to the other. See the comment here for more info : https://github.com/huggingface/datasets/issues/853#issuecomment-727872188
2021-01-24T15:16:54Z
https://github.com/huggingface/datasets/issues/1770#issuecomment-766366180
https://api.github.com/repos/huggingface/datasets/issues/1770
MDEyOklzc3VlQ29tbWVudDc2NjM2NjE4MA==
https://api.github.com/repos/huggingface/datasets/issues/comments/766366180/reactions
2021-01-24T15:16:54Z
https://api.github.com/repos/huggingface/datasets/issues/comments/766366180
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
Good to hear.
Currently I did not use map , just fetch src and tgt from the 2 dataset and merge them.
It will be a release if you can deal with it at the backend.
Thanks.
2021-01-24T23:43:54Z
https://github.com/huggingface/datasets/issues/1770#issuecomment-766461745
https://api.github.com/repos/huggingface/datasets/issues/1770
MDEyOklzc3VlQ29tbWVudDc2NjQ2MTc0NQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/766461745/reactions
2021-01-24T23:43:54Z
https://api.github.com/repos/huggingface/datasets/issues/comments/766461745
https://avatars.githubusercontent.com/u/7607120?v=4
https://api.github.com/users/world2vec/events{/privacy}
https://api.github.com/users/world2vec/followers
https://api.github.com/users/world2vec/following{/other_user}
https://api.github.com/users/world2vec/gists{/gist_id}
https://github.com/world2vec
world2vec
MDQ6VXNlcjc2MDcxMjA=
https://api.github.com/users/world2vec/orgs
https://api.github.com/users/world2vec/received_events
https://api.github.com/users/world2vec/repos
https://api.github.com/users/world2vec/starred{/owner}{/repo}
https://api.github.com/users/world2vec/subscriptions
User
https://api.github.com/users/world2vec
public
COLLABORATOR
Hi! You can rename the columns and concatenate the datasets along `axis=1` to get the desired result as follows:
```python
ds1 = ds1.rename_column("text", "src")
ds2 = ds2.rename_column("text", "tgt")
ds = datasets.concatenate_datasets(["ds1", "ds2"], axis=1)
```
2022-06-01T15:43:13Z
https://github.com/huggingface/datasets/issues/1770#issuecomment-1143774864
https://api.github.com/repos/huggingface/datasets/issues/1770
IC_kwDODunzps5ELJ6Q
https://api.github.com/repos/huggingface/datasets/issues/comments/1143774864/reactions
2022-06-01T15:43:13Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1143774864
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
to combine 2 dataset by one-one map like ds = zip(ds1, ds2):
ds1: {'text'}, ds2: {'text'}, combine ds:{'src', 'tgt'} 
or different feature:
ds1: {'src'}, ds2: {'tgt'}, combine ds:{'src', 'tgt'}