News_commentary Dataset Translation Pairs are of Incorrect Language Specified Pairs
MEMBER
Hi @anassalamah,

Could you please try with this:
```python
train_ds = load_dataset("news_commentary", lang1="ar", lang2="en", split='train[:98%]')
val_ds = load_dataset("news_commentary", lang1="ar", lang2="en", split='train[98%:]')
```
2021-04-09T09:32:20Z
https://github.com/huggingface/datasets/issues/2190#issuecomment-816554981
https://api.github.com/repos/huggingface/datasets/issues/2190
MDEyOklzc3VlQ29tbWVudDgxNjU1NDk4MQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/816554981/reactions
2021-04-09T09:32:20Z
https://api.github.com/repos/huggingface/datasets/issues/comments/816554981
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
NONE
Hello @albertvillanova, 

Thanks for the suggestion. I didn't know you could do that. however, it didn't resolve the issue

![image](https://user-images.githubusercontent.com/8571003/114169966-ec819400-993a-11eb-8a67-930f9a9b2290.png)

2021-04-09T10:54:10Z
https://github.com/huggingface/datasets/issues/2190#issuecomment-816599603
https://api.github.com/repos/huggingface/datasets/issues/2190
MDEyOklzc3VlQ29tbWVudDgxNjU5OTYwMw==
https://api.github.com/repos/huggingface/datasets/issues/comments/816599603/reactions
2021-04-09T10:54:10Z
https://api.github.com/repos/huggingface/datasets/issues/comments/816599603
https://avatars.githubusercontent.com/u/8571003?v=4
https://api.github.com/users/anassalamah/events{/privacy}
https://api.github.com/users/anassalamah/followers
https://api.github.com/users/anassalamah/following{/other_user}
https://api.github.com/users/anassalamah/gists{/gist_id}
https://github.com/anassalamah
anassalamah
MDQ6VXNlcjg1NzEwMDM=
https://api.github.com/users/anassalamah/orgs
https://api.github.com/users/anassalamah/received_events
https://api.github.com/users/anassalamah/repos
https://api.github.com/users/anassalamah/starred{/owner}{/repo}
https://api.github.com/users/anassalamah/subscriptions
User
https://api.github.com/users/anassalamah
public
I used load_dataset to load the news_commentary dataset for "ar-en" translation pairs but found translations from Arabic to Hindi.  

```
train_ds = load_dataset("news_commentary", "ar-en", split='train[:98%]')
val_ds = load_dataset("news_commentary", "ar-en", split='train[98%:]')

# filtering out examples that are not ar-en translations but ar-hi
val_ds = val_ds.filter(lambda example, indice: indice not in chain(range(1312,1327) ,range(1384,1399), range(1030,1042)), with_indices=True)
```

* I'm fairly new to using datasets so I might be doing something wrong