dataset.shuffle(keep_in_memory=True) is never allowed
CONTRIBUTOR
This seems to be fixed in #513 for the filter function, replacing `cache_file_name` with `indices_cache_file_name` in the assert. Although not for the `map()` function @thomwolf 
2020-08-18T19:08:26Z
https://github.com/huggingface/datasets/issues/514#issuecomment-675660642
https://api.github.com/repos/huggingface/datasets/issues/514
MDEyOklzc3VlQ29tbWVudDY3NTY2MDY0Mg==
https://api.github.com/repos/huggingface/datasets/issues/comments/675660642/reactions
2020-08-18T19:08:38Z
https://api.github.com/repos/huggingface/datasets/issues/comments/675660642
https://avatars.githubusercontent.com/u/24683907?v=4
https://api.github.com/users/vegarab/events{/privacy}
https://api.github.com/users/vegarab/followers
https://api.github.com/users/vegarab/following{/other_user}
https://api.github.com/users/vegarab/gists{/gist_id}
https://github.com/vegarab
vegarab
MDQ6VXNlcjI0NjgzOTA3
https://api.github.com/users/vegarab/orgs
https://api.github.com/users/vegarab/received_events
https://api.github.com/users/vegarab/repos
https://api.github.com/users/vegarab/starred{/owner}{/repo}
https://api.github.com/users/vegarab/subscriptions
User
https://api.github.com/users/vegarab
public
MEMBER
Maybe I'm a bit tired but I fail to see the issue here.

Since `cache_file_name` is `None` by default, if you set `keep_in_memory` to `True`, the assert should pass, no?
2020-08-18T19:13:14Z
https://github.com/huggingface/datasets/issues/514#issuecomment-675662774
https://api.github.com/repos/huggingface/datasets/issues/514
MDEyOklzc3VlQ29tbWVudDY3NTY2Mjc3NA==
https://api.github.com/repos/huggingface/datasets/issues/comments/675662774/reactions
2020-08-18T19:13:14Z
https://api.github.com/repos/huggingface/datasets/issues/comments/675662774
https://avatars.githubusercontent.com/u/7353373?v=4
https://api.github.com/users/thomwolf/events{/privacy}
https://api.github.com/users/thomwolf/followers
https://api.github.com/users/thomwolf/following{/other_user}
https://api.github.com/users/thomwolf/gists{/gist_id}
https://github.com/thomwolf
thomwolf
MDQ6VXNlcjczNTMzNzM=
https://api.github.com/users/thomwolf/orgs
https://api.github.com/users/thomwolf/received_events
https://api.github.com/users/thomwolf/repos
https://api.github.com/users/thomwolf/starred{/owner}{/repo}
https://api.github.com/users/thomwolf/subscriptions
User
https://api.github.com/users/thomwolf
public
CONTRIBUTOR
I failed to realise that this only applies to `shuffle()`. Whenever `keep_in_memory` is set to True, this is passed on to the `select()` function. However, if `cache_file_name` is None, it will be defined in the `shuffle()` function before it is passed on to `select()`. 

Thus, `select()` is called with `keep_in_memory=True` and a not None value for `cache_file_name`. 
This is essentially fixed in #513 

Easily reproducible:
```python
>>> import nlp
>>> data = nlp.load_dataset("cosmos_qa", split="train")
Using custom data configuration default
>>> data.shuffle(keep_in_memory=True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/vegarab/.conda/envs/torch/lib/python3.7/site-packages/nlp/arrow_dataset.py", line 1398, in shuffle
    verbose=verbose,
  File "/home/vegarab/.conda/envs/torch/lib/python3.7/site-packages/nlp/arrow_dataset.py", line 1178, in select
    ), "Please use either `keep_in_memory` or `cache_file_name` but not both."
AssertionError: Please use either `keep_in_memory` or `cache_file_name` but not both.
>>>data.select([0], keep_in_memory=True)
# No error
```
2020-08-18T19:21:40Z
https://github.com/huggingface/datasets/issues/514#issuecomment-675666647
https://api.github.com/repos/huggingface/datasets/issues/514
MDEyOklzc3VlQ29tbWVudDY3NTY2NjY0Nw==
https://api.github.com/repos/huggingface/datasets/issues/comments/675666647/reactions
2020-08-18T19:24:22Z
https://api.github.com/repos/huggingface/datasets/issues/comments/675666647
https://avatars.githubusercontent.com/u/24683907?v=4
https://api.github.com/users/vegarab/events{/privacy}
https://api.github.com/users/vegarab/followers
https://api.github.com/users/vegarab/following{/other_user}
https://api.github.com/users/vegarab/gists{/gist_id}
https://github.com/vegarab
vegarab
MDQ6VXNlcjI0NjgzOTA3
https://api.github.com/users/vegarab/orgs
https://api.github.com/users/vegarab/received_events
https://api.github.com/users/vegarab/repos
https://api.github.com/users/vegarab/starred{/owner}{/repo}
https://api.github.com/users/vegarab/subscriptions
User
https://api.github.com/users/vegarab
public
MEMBER
Oh yes ok got it thanks. Should be fixed if we are happy with #513 indeed.
2020-08-18T19:44:29Z
https://github.com/huggingface/datasets/issues/514#issuecomment-675676556
https://api.github.com/repos/huggingface/datasets/issues/514
MDEyOklzc3VlQ29tbWVudDY3NTY3NjU1Ng==
https://api.github.com/repos/huggingface/datasets/issues/comments/675676556/reactions
2020-08-18T19:44:29Z
https://api.github.com/repos/huggingface/datasets/issues/comments/675676556
https://avatars.githubusercontent.com/u/7353373?v=4
https://api.github.com/users/thomwolf/events{/privacy}
https://api.github.com/users/thomwolf/followers
https://api.github.com/users/thomwolf/following{/other_user}
https://api.github.com/users/thomwolf/gists{/gist_id}
https://github.com/thomwolf
thomwolf
MDQ6VXNlcjczNTMzNzM=
https://api.github.com/users/thomwolf/orgs
https://api.github.com/users/thomwolf/received_events
https://api.github.com/users/thomwolf/repos
https://api.github.com/users/thomwolf/starred{/owner}{/repo}
https://api.github.com/users/thomwolf/subscriptions
User
https://api.github.com/users/thomwolf
public
CONTRIBUTOR
My bad. This is actually not fixed in #513. Sorry about that...
The new `indices_cache_file_name` is set to a non-None value in the new `shuffle()` as well. 

The buffer and caching mechanisms used in the `select()` function are too intricate for me to understand why the check is there at all. I've removed it in my local build and it seems to be working fine for my project, without really considering other implications of the change. 


2020-08-19T11:36:02Z
https://github.com/huggingface/datasets/issues/514#issuecomment-676197996
https://api.github.com/repos/huggingface/datasets/issues/514
MDEyOklzc3VlQ29tbWVudDY3NjE5Nzk5Ng==
https://api.github.com/repos/huggingface/datasets/issues/comments/676197996/reactions
2020-08-19T11:36:02Z
https://api.github.com/repos/huggingface/datasets/issues/comments/676197996
https://avatars.githubusercontent.com/u/24683907?v=4
https://api.github.com/users/vegarab/events{/privacy}
https://api.github.com/users/vegarab/followers
https://api.github.com/users/vegarab/following{/other_user}
https://api.github.com/users/vegarab/gists{/gist_id}
https://github.com/vegarab
vegarab
MDQ6VXNlcjI0NjgzOTA3
https://api.github.com/users/vegarab/orgs
https://api.github.com/users/vegarab/received_events
https://api.github.com/users/vegarab/repos
https://api.github.com/users/vegarab/starred{/owner}{/repo}
https://api.github.com/users/vegarab/subscriptions
User
https://api.github.com/users/vegarab
public
MEMBER
Ok I'll investigate and add a series of tests on the `keep_in_memory=True` settings which is under-tested atm
2020-08-19T12:18:46Z
https://github.com/huggingface/datasets/issues/514#issuecomment-676256821
https://api.github.com/repos/huggingface/datasets/issues/514
MDEyOklzc3VlQ29tbWVudDY3NjI1NjgyMQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/676256821/reactions
2020-08-19T12:18:46Z
https://api.github.com/repos/huggingface/datasets/issues/comments/676256821
https://avatars.githubusercontent.com/u/7353373?v=4
https://api.github.com/users/thomwolf/events{/privacy}
https://api.github.com/users/thomwolf/followers
https://api.github.com/users/thomwolf/following{/other_user}
https://api.github.com/users/thomwolf/gists{/gist_id}
https://github.com/thomwolf
thomwolf
MDQ6VXNlcjczNTMzNzM=
https://api.github.com/users/thomwolf/orgs
https://api.github.com/users/thomwolf/received_events
https://api.github.com/users/thomwolf/repos
https://api.github.com/users/thomwolf/starred{/owner}{/repo}
https://api.github.com/users/thomwolf/subscriptions
User
https://api.github.com/users/thomwolf
public
NONE
Hey, still seeing this issue with the latest version.
2021-07-23T18:07:10Z
https://github.com/huggingface/datasets/issues/514#issuecomment-885811187
https://api.github.com/repos/huggingface/datasets/issues/514
IC_kwDODunzps40zGfz
https://api.github.com/repos/huggingface/datasets/issues/comments/885811187/reactions
2021-07-23T18:07:10Z
https://api.github.com/repos/huggingface/datasets/issues/comments/885811187
https://avatars.githubusercontent.com/u/8812459?v=4
https://api.github.com/users/epwalsh/events{/privacy}
https://api.github.com/users/epwalsh/followers
https://api.github.com/users/epwalsh/following{/other_user}
https://api.github.com/users/epwalsh/gists{/gist_id}
https://github.com/epwalsh
epwalsh
MDQ6VXNlcjg4MTI0NTk=
https://api.github.com/users/epwalsh/orgs
https://api.github.com/users/epwalsh/received_events
https://api.github.com/users/epwalsh/repos
https://api.github.com/users/epwalsh/starred{/owner}{/repo}
https://api.github.com/users/epwalsh/subscriptions
User
https://api.github.com/users/epwalsh
public
NONE
The same :(
2022-09-19T03:18:40Z
https://github.com/huggingface/datasets/issues/514#issuecomment-1250522184
https://api.github.com/repos/huggingface/datasets/issues/514
IC_kwDODunzps5KiXRI
https://api.github.com/repos/huggingface/datasets/issues/comments/1250522184/reactions
2022-09-19T03:18:40Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1250522184
https://avatars.githubusercontent.com/u/45794881?v=4
https://api.github.com/users/koren-v/events{/privacy}
https://api.github.com/users/koren-v/followers
https://api.github.com/users/koren-v/following{/other_user}
https://api.github.com/users/koren-v/gists{/gist_id}
https://github.com/koren-v
koren-v
MDQ6VXNlcjQ1Nzk0ODgx
https://api.github.com/users/koren-v/orgs
https://api.github.com/users/koren-v/received_events
https://api.github.com/users/koren-v/repos
https://api.github.com/users/koren-v/starred{/owner}{/repo}
https://api.github.com/users/koren-v/subscriptions
User
https://api.github.com/users/koren-v
public
COLLABORATOR
These are the steps needed to fix this issue:
1. add the following check to `Dataset.shuffle`:
```python
if keep_in_memory and indices_cache_file_name is not None:
    raise ValueError("Please use either `keep_in_memory` or `indices_cache_file_name` but not both.")
```
2. set `indices_cache_file_name` to `None` if `keep_in_memory` is True in the call to `select`
3. add a test with `shuffle(keep_in_memory=True)`
2022-10-04T17:43:13Z
https://github.com/huggingface/datasets/issues/514#issuecomment-1267344346
https://api.github.com/repos/huggingface/datasets/issues/514
IC_kwDODunzps5LiiPa
https://api.github.com/repos/huggingface/datasets/issues/comments/1267344346/reactions
2022-10-04T17:43:13Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1267344346
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
CONTRIBUTOR
Hi @mariosasko , I have opened this PR #5082 
2022-10-06T11:33:03Z
https://github.com/huggingface/datasets/issues/514#issuecomment-1269863844
https://api.github.com/repos/huggingface/datasets/issues/514
IC_kwDODunzps5LsJWk
https://api.github.com/repos/huggingface/datasets/issues/comments/1269863844/reactions
2022-10-06T11:33:03Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1269863844
https://avatars.githubusercontent.com/u/66799406?v=4
https://api.github.com/users/Mustapha-AJEGHRIR/events{/privacy}
https://api.github.com/users/Mustapha-AJEGHRIR/followers
https://api.github.com/users/Mustapha-AJEGHRIR/following{/other_user}
https://api.github.com/users/Mustapha-AJEGHRIR/gists{/gist_id}
https://github.com/Mustapha-AJEGHRIR
Mustapha-AJEGHRIR
MDQ6VXNlcjY2Nzk5NDA2
https://api.github.com/users/Mustapha-AJEGHRIR/orgs
https://api.github.com/users/Mustapha-AJEGHRIR/received_events
https://api.github.com/users/Mustapha-AJEGHRIR/repos
https://api.github.com/users/Mustapha-AJEGHRIR/starred{/owner}{/repo}
https://api.github.com/users/Mustapha-AJEGHRIR/subscriptions
User
https://api.github.com/users/Mustapha-AJEGHRIR
public
As of commit ef4aac2, the usage of the parameter `keep_in_memory=True` is never possible: `dataset.select(keep_in_memory=True)`

The commit added the lines
```python
# lines 994-996 in src/nlp/arrow_dataset.py
       assert (
            not keep_in_memory or cache_file_name is None
        ), "Please use either `keep_in_memory` or `cache_file_name` but not both."
```

This affects both `shuffle()` as `select()` is a sub-routine, and `map()` that has the same check. 

I'd love to fix this myself, but unsure what the intention of the assert is given the rest of the logic in the function concerning `ccache_file_name` and `keep_in_memory`.