Provide a way to write a streamed dataset to the disk
COLLABORATOR
Yes, I agree this feature is much needed. We could do something similar to what TF does (https://www.tensorflow.org/api_docs/python/tf/data/Dataset#cache). 

Ideally, if the entire streamed dataset is consumed/cached, the generated cache should be reusable for the Arrow dataset.
2021-10-29T11:08:03Z
https://github.com/huggingface/datasets/issues/3142#issuecomment-954655645
https://api.github.com/repos/huggingface/datasets/issues/3142
IC_kwDODunzps445uOd
https://api.github.com/repos/huggingface/datasets/issues/comments/954655645/reactions
2021-10-29T11:14:39Z
https://api.github.com/repos/huggingface/datasets/issues/comments/954655645
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
NONE
@mariosasko Hi big brother,any update on this? It's 2024 with large streamed dataset loading consume too much time(exp.  2day..), really need this feature for what TF does
2024-01-12T07:26:42Z
https://github.com/huggingface/datasets/issues/3142#issuecomment-1888559980
https://api.github.com/repos/huggingface/datasets/issues/3142
IC_kwDODunzps5wkSNs
https://api.github.com/repos/huggingface/datasets/issues/comments/1888559980/reactions
2024-01-12T07:26:42Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1888559980
https://avatars.githubusercontent.com/u/115082138?v=4
https://api.github.com/users/nsc2cn/events{/privacy}
https://api.github.com/users/nsc2cn/followers
https://api.github.com/users/nsc2cn/following{/other_user}
https://api.github.com/users/nsc2cn/gists{/gist_id}
https://github.com/nsc2cn
nsc2cn
U_kgDOBtwDmg
https://api.github.com/users/nsc2cn/orgs
https://api.github.com/users/nsc2cn/received_events
https://api.github.com/users/nsc2cn/repos
https://api.github.com/users/nsc2cn/starred{/owner}{/repo}
https://api.github.com/users/nsc2cn/subscriptions
User
https://api.github.com/users/nsc2cn
public
**Is your feature request related to a problem? Please describe.**

The streaming mode allows to get the 100 first rows of a dataset very quickly. But it does not cache the answer, so a posterior call to get the same 100 rows will send a request to the server again and again.

**Describe the solution you'd like**

Provide a way to write the streamed rows of a dataset on the disk, and to load from it later.

**Describe alternatives you've considered**

Provide a third mode: `lazy`, which would use the local cache for the data that have already been fetched previously, and use streaming to get the rest of the requested data. 
