Safely load datasets by disabling execution of dataset loading script
MEMBER
great idea IMO

this could be a `trust_remote_code=True` flag like in transformers. We could also default to loading the Parquet conversion rather than executing code (for dataset repos that have both)
2023-11-13T10:10:35Z
https://github.com/huggingface/datasets/issues/6400#issuecomment-1807855083
https://api.github.com/repos/huggingface/datasets/issues/6400
IC_kwDODunzps5rwa3r
https://api.github.com/repos/huggingface/datasets/issues/comments/1807855083/reactions
2023-11-13T10:13:22Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1807855083
https://avatars.githubusercontent.com/u/326577?v=4
https://api.github.com/users/julien-c/events{/privacy}
https://api.github.com/users/julien-c/followers
https://api.github.com/users/julien-c/following{/other_user}
https://api.github.com/users/julien-c/gists{/gist_id}
https://github.com/julien-c
julien-c
MDQ6VXNlcjMyNjU3Nw==
https://api.github.com/users/julien-c/orgs
https://api.github.com/users/julien-c/received_events
https://api.github.com/users/julien-c/repos
https://api.github.com/users/julien-c/starred{/owner}{/repo}
https://api.github.com/users/julien-c/subscriptions
User
https://api.github.com/users/julien-c
public
CONTRIBUTOR
@julien-c that would be great!
2023-11-13T23:55:59Z
https://github.com/huggingface/datasets/issues/6400#issuecomment-1809319962
https://api.github.com/repos/huggingface/datasets/issues/6400
IC_kwDODunzps5r2Aga
https://api.github.com/repos/huggingface/datasets/issues/comments/1809319962/reactions
2023-11-13T23:55:59Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1809319962
https://avatars.githubusercontent.com/u/43149077?v=4
https://api.github.com/users/dakinggg/events{/privacy}
https://api.github.com/users/dakinggg/followers
https://api.github.com/users/dakinggg/following{/other_user}
https://api.github.com/users/dakinggg/gists{/gist_id}
https://github.com/dakinggg
dakinggg
MDQ6VXNlcjQzMTQ5MDc3
https://api.github.com/users/dakinggg/orgs
https://api.github.com/users/dakinggg/received_events
https://api.github.com/users/dakinggg/repos
https://api.github.com/users/dakinggg/starred{/owner}{/repo}
https://api.github.com/users/dakinggg/subscriptions
User
https://api.github.com/users/dakinggg
public
MEMBER
We added the `trust_remote_code` argument to `load_dataset()` in `datasets` 2.16:
- in the future users will have to pass trust_remote_code=True to use datasets with a script
- for now we just show a warning when a dataset script is used
- we fallback on the Hugging Face Parquet exports when possible (to keep compatibility with old datasets with scripts)

So feel free to use `trust_remote_code=False` in the meantime to disable loading from dataset loading scripts :)
2024-01-02T18:18:08Z
https://github.com/huggingface/datasets/issues/6400#issuecomment-1874380073
https://api.github.com/repos/huggingface/datasets/issues/6400
IC_kwDODunzps5vuMUp
https://api.github.com/repos/huggingface/datasets/issues/comments/1874380073/reactions
2024-01-02T18:18:08Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1874380073
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
Passing `trust_remote_code=True` explicitly is now mandatory to load a dataset with a script since https://github.com/huggingface/datasets/pull/6954
2024-06-13T13:02:08Z
https://github.com/huggingface/datasets/issues/6400#issuecomment-2165606919
https://api.github.com/repos/huggingface/datasets/issues/6400
IC_kwDODunzps6BFIoH
https://api.github.com/repos/huggingface/datasets/issues/comments/2165606919/reactions
2024-06-13T13:02:08Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2165606919
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
### Feature request

Is there a way to disable execution of dataset loading script using `load_dataset`? This is a security vulnerability that could lead to arbitrary code execution. 

Any suggested workarounds are welcome as well. 

### Motivation

This is a security vulnerability that could lead to arbitrary code execution. 

### Your contribution

n/a