Support WebDataset containing file basenames with dots
NONE
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_6888). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
2024-05-09T08:28:11Z
https://github.com/huggingface/datasets/pull/6888#issuecomment-2102199781
https://api.github.com/repos/huggingface/datasets/issues/6888
IC_kwDODunzps59TQXl
https://api.github.com/repos/huggingface/datasets/issues/comments/2102199781/reactions
2024-05-09T08:28:11Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2102199781
https://avatars.githubusercontent.com/u/99929124?v=4
https://api.github.com/users/HuggingFaceDocBuilderDev/events{/privacy}
https://api.github.com/users/HuggingFaceDocBuilderDev/followers
https://api.github.com/users/HuggingFaceDocBuilderDev/following{/other_user}
https://api.github.com/users/HuggingFaceDocBuilderDev/gists{/gist_id}
https://github.com/HuggingFaceDocBuilderDev
HuggingFaceDocBuilderDev
U_kgDOBfTMJA
https://api.github.com/users/HuggingFaceDocBuilderDev/orgs
https://api.github.com/users/HuggingFaceDocBuilderDev/received_events
https://api.github.com/users/HuggingFaceDocBuilderDev/repos
https://api.github.com/users/HuggingFaceDocBuilderDev/starred{/owner}{/repo}
https://api.github.com/users/HuggingFaceDocBuilderDev/subscriptions
User
https://api.github.com/users/HuggingFaceDocBuilderDev
public
MEMBER
I think webdataset splits the file name and extension using the first dot no ?

https://github.com/webdataset/webdataset/blob/945b251a872ec0d337be8f9ea17a9c5b0d017ff3/webdataset/tariterators.py#L226

links to this function that splits on first dot

```python

def base_plus_ext(path):
    """Split off all file extensions.

    Returns base, allext.

    Args:
        path: path with extensions

    Returns:
        path with all extensions removed
    """
    match = re.match(r"^((?:.*/|)[^.]+)[.]([^/]*)$", path)
    if not match:
        return None, None
    return match.group(1), match.group(2)
```
2024-05-10T10:14:38Z
https://github.com/huggingface/datasets/pull/6888#issuecomment-2104342995
https://api.github.com/repos/huggingface/datasets/issues/6888
IC_kwDODunzps59bbnT
https://api.github.com/repos/huggingface/datasets/issues/comments/2104342995/reactions
2024-05-10T10:14:38Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2104342995
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
So maybe the original issue is actually due to one of the files containing a dot in its file name that is not for the extension

```python
>>> base_plus_ext("15_Cohen_1-s2.0-S0929664620300449-gr3_lrg-b.png")
('15_Cohen_1-s2', '0-S0929664620300449-gr3_lrg-b.png')
```
2024-05-10T10:16:28Z
https://github.com/huggingface/datasets/pull/6888#issuecomment-2104345834
https://api.github.com/repos/huggingface/datasets/issues/6888
IC_kwDODunzps59bcTq
https://api.github.com/repos/huggingface/datasets/issues/comments/2104345834/reactions
2024-05-10T10:17:27Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2104345834
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
Thanks for your review, @lhoestq.

I was not aware that `webdataset` requires filenames without dots in their basenames.
2024-05-10T13:47:39Z
https://github.com/huggingface/datasets/pull/6888#issuecomment-2104643033
https://api.github.com/repos/huggingface/datasets/issues/6888
IC_kwDODunzps59ck3Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2104643033/reactions
2024-05-10T13:47:39Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2104643033
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
MEMBER
I they can have dots for the extension (that becomes the column name) but not in the key used to group files into samples
2024-05-10T13:48:55Z
https://github.com/huggingface/datasets/pull/6888#issuecomment-2104645065
https://api.github.com/repos/huggingface/datasets/issues/6888
IC_kwDODunzps59clXJ
https://api.github.com/repos/huggingface/datasets/issues/comments/2104645065/reactions
2024-05-10T13:49:29Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2104645065
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
Support WebDataset containing file basenames with dots.

Fix #6880.