Support fsspec 2023.10.0
NONE
Hi @albertvillanova @lhoestq 

I believe the pull request that pins the fsspec version (https://github.com/huggingface/datasets/pull/6331) was merged by mistake. Another fix for the issue was merged on the same day an hour apart. See https://github.com/huggingface/datasets/pull/6334

I'm now having an issue in my project where I can't use newer versions of fsspec.

Can we remove the pin?

Have a nice day! :)
2024-02-07T11:14:25Z
https://github.com/huggingface/datasets/issues/6333#issuecomment-1931822827
https://api.github.com/repos/huggingface/datasets/issues/6333
IC_kwDODunzps5zJUbr
https://api.github.com/repos/huggingface/datasets/issues/comments/1931822827/reactions
2024-02-07T11:15:29Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1931822827
https://avatars.githubusercontent.com/u/20381749?v=4
https://api.github.com/users/tomscholz/events{/privacy}
https://api.github.com/users/tomscholz/followers
https://api.github.com/users/tomscholz/following{/other_user}
https://api.github.com/users/tomscholz/gists{/gist_id}
https://github.com/tomscholz
tomscholz
MDQ6VXNlcjIwMzgxNzQ5
https://api.github.com/users/tomscholz/orgs
https://api.github.com/users/tomscholz/received_events
https://api.github.com/users/tomscholz/repos
https://api.github.com/users/tomscholz/starred{/owner}{/repo}
https://api.github.com/users/tomscholz/subscriptions
User
https://api.github.com/users/tomscholz
public
MEMBER
Hi @tomscholz,

Thanks for pointing this out. I think you are right.

I am doing some cross-checks and fixing it. 
2024-02-07T12:23:44Z
https://github.com/huggingface/datasets/issues/6333#issuecomment-1931937479
https://api.github.com/repos/huggingface/datasets/issues/6333
IC_kwDODunzps5zJwbH
https://api.github.com/repos/huggingface/datasets/issues/comments/1931937479/reactions
2024-02-07T12:24:29Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1931937479
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
MEMBER
Hi again, @tomscholz.

After a more cautious investigation, I think the pin is OK because there are other reasons for it. Chronologically:
- #6331 
- #6334
- #6336 
- #6337 

The reason is that after version 2023.10.0, they changed again the behavior of their `glob` function. See: https://github.com/huggingface/datasets/pull/6337#issuecomment-1774930135
We are working on our side to support both previous and new glob behavior.

Note:
- First pin was < 2023.10.0
- Last pin is <= 2023.10.0
2024-02-07T12:31:02Z
https://github.com/huggingface/datasets/issues/6333#issuecomment-1931949706
https://api.github.com/repos/huggingface/datasets/issues/6333
IC_kwDODunzps5zJzaK
https://api.github.com/repos/huggingface/datasets/issues/comments/1931949706/reactions
2024-02-07T12:37:35Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1931949706
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
MEMBER
Fixed by #6334 and #6336.
2024-02-07T12:39:58Z
https://github.com/huggingface/datasets/issues/6333#issuecomment-1931964416
https://api.github.com/repos/huggingface/datasets/issues/6333
IC_kwDODunzps5zJ3AA
https://api.github.com/repos/huggingface/datasets/issues/comments/1931964416/reactions
2024-02-07T12:39:58Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1931964416
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
Once root issue is fixed, remove temporary pin of fsspec < 2023.10.0 introduced by:
- #6331

Related to issue:
- #6330

As @ZachNagengast suggested, the issue might be related to:
- https://github.com/fsspec/filesystem_spec/pull/1381