Efficient ways to iterate the dataset
CONTRIBUTOR
It seems that selecting a subset of colums directly from the dataset, i.e., dataset["column"], is slow.
2021-01-24T09:26:21Z
https://github.com/huggingface/datasets/issues/1775#issuecomment-766317289
https://api.github.com/repos/huggingface/datasets/issues/1775
MDEyOklzc3VlQ29tbWVudDc2NjMxNzI4OQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/766317289/reactions
2021-01-24T09:26:21Z
https://api.github.com/repos/huggingface/datasets/issues/comments/766317289
https://avatars.githubusercontent.com/u/11826803?v=4
https://api.github.com/users/zhongpeixiang/events{/privacy}
https://api.github.com/users/zhongpeixiang/followers
https://api.github.com/users/zhongpeixiang/following{/other_user}
https://api.github.com/users/zhongpeixiang/gists{/gist_id}
https://github.com/zhongpeixiang
zhongpeixiang
MDQ6VXNlcjExODI2ODAz
https://api.github.com/users/zhongpeixiang/orgs
https://api.github.com/users/zhongpeixiang/received_events
https://api.github.com/users/zhongpeixiang/repos
https://api.github.com/users/zhongpeixiang/starred{/owner}{/repo}
https://api.github.com/users/zhongpeixiang/subscriptions
User
https://api.github.com/users/zhongpeixiang
public
CONTRIBUTOR
I was wrong, ```dataset["column"]``` is fast.
2021-01-24T09:50:39Z
https://github.com/huggingface/datasets/issues/1775#issuecomment-766320196
https://api.github.com/repos/huggingface/datasets/issues/1775
MDEyOklzc3VlQ29tbWVudDc2NjMyMDE5Ng==
https://api.github.com/repos/huggingface/datasets/issues/comments/766320196/reactions
2021-01-24T09:50:39Z
https://api.github.com/repos/huggingface/datasets/issues/comments/766320196
https://avatars.githubusercontent.com/u/11826803?v=4
https://api.github.com/users/zhongpeixiang/events{/privacy}
https://api.github.com/users/zhongpeixiang/followers
https://api.github.com/users/zhongpeixiang/following{/other_user}
https://api.github.com/users/zhongpeixiang/gists{/gist_id}
https://github.com/zhongpeixiang
zhongpeixiang
MDQ6VXNlcjExODI2ODAz
https://api.github.com/users/zhongpeixiang/orgs
https://api.github.com/users/zhongpeixiang/received_events
https://api.github.com/users/zhongpeixiang/repos
https://api.github.com/users/zhongpeixiang/starred{/owner}{/repo}
https://api.github.com/users/zhongpeixiang/subscriptions
User
https://api.github.com/users/zhongpeixiang
public
For a large dataset that does not fits the memory, how can I select only a subset of features from each example?

If I iterate over the dataset and then select the subset of features one by one, the resulted memory usage will be huge. Any ways to solve this?

Thanks