Dataset loading script method does not work with .pyc file
COLLABORATOR
Before dynamically loading `.py` scripts with `importlib.import_module`, we also parse their contents to check imports, which is tricky to implement for binary `.pyc` files (requires parsing bytecode), so I don't think this is something we want to support (unless more users request it ofc) as this use case is a bit too specific.

@lhoestq What's your opinion on this?
2023-08-30T16:54:26Z
https://github.com/huggingface/datasets/issues/6193#issuecomment-1699532471
https://api.github.com/repos/huggingface/datasets/issues/6193
IC_kwDODunzps5lTM63
https://api.github.com/repos/huggingface/datasets/issues/comments/1699532471/reactions
2023-08-30T16:54:26Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1699532471
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
MEMBER
> Before dynamically loading .py scripts with importlib.import_module, we also parse their contents to check imports, which is tricky to implement for binary .pyc files (requires parsing bytecode), so I don't think this is something we want to support (unless more users request it ofc) as this use case is a bit too specific.

Yes indeed. Though you can use a .py that imports a package that contains your .pyc code and that you previously installed
2023-08-31T09:13:36Z
https://github.com/huggingface/datasets/issues/6193#issuecomment-1700664268
https://api.github.com/repos/huggingface/datasets/issues/6193
IC_kwDODunzps5lXhPM
https://api.github.com/repos/huggingface/datasets/issues/comments/1700664268/reactions
2023-08-31T09:13:36Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1700664268
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
Hi @lhoestq ,
Could you share some example code related to the approach that you are suggesting? 
2023-08-31T19:47:28Z
https://github.com/huggingface/datasets/issues/6193#issuecomment-1701688645
https://api.github.com/repos/huggingface/datasets/issues/6193
IC_kwDODunzps5lbbVF
https://api.github.com/repos/huggingface/datasets/issues/comments/1701688645/reactions
2023-08-31T19:47:28Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1701688645
https://avatars.githubusercontent.com/u/43389071?v=4
https://api.github.com/users/riteshkumarumassedu/events{/privacy}
https://api.github.com/users/riteshkumarumassedu/followers
https://api.github.com/users/riteshkumarumassedu/following{/other_user}
https://api.github.com/users/riteshkumarumassedu/gists{/gist_id}
https://github.com/riteshkumarumassedu
riteshkumarumassedu
MDQ6VXNlcjQzMzg5MDcx
https://api.github.com/users/riteshkumarumassedu/orgs
https://api.github.com/users/riteshkumarumassedu/received_events
https://api.github.com/users/riteshkumarumassedu/repos
https://api.github.com/users/riteshkumarumassedu/starred{/owner}{/repo}
https://api.github.com/users/riteshkumarumassedu/subscriptions
User
https://api.github.com/users/riteshkumarumassedu
public
### Describe the bug

The huggingface dataset library specifically looks for ‘.py’ file while loading the dataset using loading script approach and it does not work with ‘.pyc’ file.
While deploying in production, it becomes an issue when we are restricted to use only .pyc files. Is there any work around for this ?

### Steps to reproduce the bug

1. Create a dataset  loading script to read the custom data.
2. compile the code to make sure that .pyc file is created 
3. Delete the loading script and re-run the code. Usually, python should make use of complied .pyc files. However, in this case, the dataset library errors out with the message that it's unable to find the data loader loading script.

### Expected behavior

The code should make use of .pyc file and run without any error.

### Environment info

NA