load_dataset broken in 2.21.0
NONE
There seems to be a PR related to the load_dataset path that went into 2.21.0 -- https://github.com/huggingface/datasets/pull/6862/files

Taking a look at it now
2024-08-16T15:00:41Z
https://github.com/huggingface/datasets/issues/7107#issuecomment-2293672723
https://api.github.com/repos/huggingface/datasets/issues/7107
IC_kwDODunzps6ItqsT
https://api.github.com/repos/huggingface/datasets/issues/comments/2293672723/reactions
2024-08-16T15:00:41Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2293672723
https://avatars.githubusercontent.com/u/1911631?v=4
https://api.github.com/users/anjor/events{/privacy}
https://api.github.com/users/anjor/followers
https://api.github.com/users/anjor/following{/other_user}
https://api.github.com/users/anjor/gists{/gist_id}
https://github.com/anjor
anjor
MDQ6VXNlcjE5MTE2MzE=
https://api.github.com/users/anjor/orgs
https://api.github.com/users/anjor/received_events
https://api.github.com/users/anjor/repos
https://api.github.com/users/anjor/starred{/owner}{/repo}
https://api.github.com/users/anjor/subscriptions
User
https://api.github.com/users/anjor
public
NONE
+1

Downgrading to 2.20.0 fixed my issue, hopefully helpful for others.
2024-08-16T15:47:29Z
https://github.com/huggingface/datasets/issues/7107#issuecomment-2293746891
https://api.github.com/repos/huggingface/datasets/issues/7107
IC_kwDODunzps6It8zL
https://api.github.com/repos/huggingface/datasets/issues/comments/2293746891/reactions
2024-08-16T15:47:29Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2293746891
https://avatars.githubusercontent.com/u/25864937?v=4
https://api.github.com/users/CShorten/events{/privacy}
https://api.github.com/users/CShorten/followers
https://api.github.com/users/CShorten/following{/other_user}
https://api.github.com/users/CShorten/gists{/gist_id}
https://github.com/CShorten
CShorten
MDQ6VXNlcjI1ODY0OTM3
https://api.github.com/users/CShorten/orgs
https://api.github.com/users/CShorten/received_events
https://api.github.com/users/CShorten/repos
https://api.github.com/users/CShorten/starred{/owner}{/repo}
https://api.github.com/users/CShorten/subscriptions
User
https://api.github.com/users/CShorten
public
NONE
I tried adding a simple test to `test_load.py` with the alpaca eval dataset but the test didn't fail :(. 

So looks like this might have something to do with the environment? 
2024-08-17T17:42:09Z
https://github.com/huggingface/datasets/issues/7107#issuecomment-2294924094
https://api.github.com/repos/huggingface/datasets/issues/7107
IC_kwDODunzps6IycM-
https://api.github.com/repos/huggingface/datasets/issues/comments/2294924094/reactions
2024-08-17T17:42:09Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2294924094
https://avatars.githubusercontent.com/u/1911631?v=4
https://api.github.com/users/anjor/events{/privacy}
https://api.github.com/users/anjor/followers
https://api.github.com/users/anjor/following{/other_user}
https://api.github.com/users/anjor/gists{/gist_id}
https://github.com/anjor
anjor
MDQ6VXNlcjE5MTE2MzE=
https://api.github.com/users/anjor/orgs
https://api.github.com/users/anjor/received_events
https://api.github.com/users/anjor/repos
https://api.github.com/users/anjor/starred{/owner}{/repo}
https://api.github.com/users/anjor/subscriptions
User
https://api.github.com/users/anjor
public
MEMBER
There was an issue with the script of the "tatsu-lab/alpaca_eval" dataset.

I was fixed with this PR: 
- [Fix FileNotFoundError](https://huggingface.co/datasets/tatsu-lab/alpaca_eval/discussions/2)

It should work now if you retry to load the dataset.
2024-08-18T09:27:13Z
https://github.com/huggingface/datasets/issues/7107#issuecomment-2295193028
https://api.github.com/repos/huggingface/datasets/issues/7107
IC_kwDODunzps6Izd3E
https://api.github.com/repos/huggingface/datasets/issues/comments/2295193028/reactions
2024-08-18T09:28:43Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2295193028
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
### Describe the bug

`eval_set = datasets.load_dataset("tatsu-lab/alpaca_eval", "alpaca_eval_gpt4_baseline", trust_remote_code=True)` 
used to work till 2.20.0 but doesn't work in 2.21.0

In 2.20.0:
![Screenshot 2024-08-16 at 3 57 10 PM](https://github.com/user-attachments/assets/0516489b-8187-486d-bee8-88af3381dee9)


in 2.21.0:
![Screenshot 2024-08-16 at 3 57 24 PM](https://github.com/user-attachments/assets/bc257570-f461-41e4-8717-90a69ed7c24f)


### Steps to reproduce the bug

1. Spin up a new google collab
2. `pip install datasets==2.21.0`
3. `import datasets`
4. `eval_set = datasets.load_dataset("tatsu-lab/alpaca_eval", "alpaca_eval_gpt4_baseline", trust_remote_code=True)` 
5. Will throw an error.



### Expected behavior

Try steps 1-5 again but replace datasets version with 2.20.0, it will work

### Environment info



- `datasets` version: 2.21.0
- Platform: Linux-6.1.85+-x86_64-with-glibc2.35
- Python version: 3.10.12
- `huggingface_hub` version: 0.23.5
- PyArrow version: 17.0.0
- Pandas version: 2.1.4
- `fsspec` version: 2024.5.0
