Return the name of the currently loaded file in the load_dataset function.
COLLABORATOR
Implementing this makes sense (e.g., `tensorflow_datasets`' imagefolder returns image filenames). Also, in Datasets 3.0, we plan only to store the bytes of an image/audio, not its path, so this feature would be useful when the path info is still needed.
2023-05-08T17:10:26Z
https://github.com/huggingface/datasets/issues/5806#issuecomment-1538740917
https://api.github.com/repos/huggingface/datasets/issues/5806
IC_kwDODunzps5bt1K1
https://api.github.com/repos/huggingface/datasets/issues/comments/1538740917/reactions
2023-05-08T17:10:26Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1538740917
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
NONE
Hey @mariosasko, Can I work on this issue, this one seems interesting to implement. I have contributed to jupyterlab recently, and would love to contribute here as well. 
2023-05-16T10:49:30Z
https://github.com/huggingface/datasets/issues/5806#issuecomment-1549431794
https://api.github.com/repos/huggingface/datasets/issues/5806
IC_kwDODunzps5cWnPy
https://api.github.com/repos/huggingface/datasets/issues/comments/1549431794/reactions
2023-05-16T10:49:30Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1549431794
https://avatars.githubusercontent.com/u/49894149?v=4
https://api.github.com/users/tsabbir96/events{/privacy}
https://api.github.com/users/tsabbir96/followers
https://api.github.com/users/tsabbir96/following{/other_user}
https://api.github.com/users/tsabbir96/gists{/gist_id}
https://github.com/tsabbir96
tsabbir96
MDQ6VXNlcjQ5ODk0MTQ5
https://api.github.com/users/tsabbir96/orgs
https://api.github.com/users/tsabbir96/received_events
https://api.github.com/users/tsabbir96/repos
https://api.github.com/users/tsabbir96/starred{/owner}{/repo}
https://api.github.com/users/tsabbir96/subscriptions
User
https://api.github.com/users/tsabbir96
public
MEMBER
@tsabbir96 if you are planning to start working on this, you can take on this issue by writing a comment with only the keyword: #self-assign
2023-05-16T13:02:41Z
https://github.com/huggingface/datasets/issues/5806#issuecomment-1549623326
https://api.github.com/repos/huggingface/datasets/issues/5806
IC_kwDODunzps5cXWAe
https://api.github.com/repos/huggingface/datasets/issues/comments/1549623326/reactions
2023-05-16T13:03:27Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1549623326
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
NONE
#self-assign
2023-05-16T16:26:09Z
https://github.com/huggingface/datasets/issues/5806#issuecomment-1549992830
https://api.github.com/repos/huggingface/datasets/issues/5806
IC_kwDODunzps5cYwN-
https://api.github.com/repos/huggingface/datasets/issues/comments/1549992830/reactions
2023-05-16T16:26:09Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1549992830
https://avatars.githubusercontent.com/u/49894149?v=4
https://api.github.com/users/tsabbir96/events{/privacy}
https://api.github.com/users/tsabbir96/followers
https://api.github.com/users/tsabbir96/following{/other_user}
https://api.github.com/users/tsabbir96/gists{/gist_id}
https://github.com/tsabbir96
tsabbir96
MDQ6VXNlcjQ5ODk0MTQ5
https://api.github.com/users/tsabbir96/orgs
https://api.github.com/users/tsabbir96/received_events
https://api.github.com/users/tsabbir96/repos
https://api.github.com/users/tsabbir96/starred{/owner}{/repo}
https://api.github.com/users/tsabbir96/subscriptions
User
https://api.github.com/users/tsabbir96
public
NONE
@albertvillanova thank you for letting me contribute here. 
@albertvillanova @mariosasko As I am totally new to this repo, could you tell me something more about this issue or perhaps give me some idea on how I can proceed with it? Thanks!
2023-05-16T16:29:34Z
https://github.com/huggingface/datasets/issues/5806#issuecomment-1549997661
https://api.github.com/repos/huggingface/datasets/issues/5806
IC_kwDODunzps5cYxZd
https://api.github.com/repos/huggingface/datasets/issues/comments/1549997661/reactions
2023-05-16T16:29:34Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1549997661
https://avatars.githubusercontent.com/u/49894149?v=4
https://api.github.com/users/tsabbir96/events{/privacy}
https://api.github.com/users/tsabbir96/followers
https://api.github.com/users/tsabbir96/following{/other_user}
https://api.github.com/users/tsabbir96/gists{/gist_id}
https://github.com/tsabbir96
tsabbir96
MDQ6VXNlcjQ5ODk0MTQ5
https://api.github.com/users/tsabbir96/orgs
https://api.github.com/users/tsabbir96/received_events
https://api.github.com/users/tsabbir96/repos
https://api.github.com/users/tsabbir96/starred{/owner}{/repo}
https://api.github.com/users/tsabbir96/subscriptions
User
https://api.github.com/users/tsabbir96
public
NONE
Hello there, is this issue resolved? @tsabbir96 are you still working on it? Otherwise I would love to give it a try
2023-07-25T21:43:38Z
https://github.com/huggingface/datasets/issues/5806#issuecomment-1650602290
https://api.github.com/repos/huggingface/datasets/issues/5806
IC_kwDODunzps5iYjEy
https://api.github.com/repos/huggingface/datasets/issues/comments/1650602290/reactions
2023-07-25T21:43:38Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1650602290
https://avatars.githubusercontent.com/u/69953243?v=4
https://api.github.com/users/EduardoPach/events{/privacy}
https://api.github.com/users/EduardoPach/followers
https://api.github.com/users/EduardoPach/following{/other_user}
https://api.github.com/users/EduardoPach/gists{/gist_id}
https://github.com/EduardoPach
EduardoPach
MDQ6VXNlcjY5OTUzMjQz
https://api.github.com/users/EduardoPach/orgs
https://api.github.com/users/EduardoPach/received_events
https://api.github.com/users/EduardoPach/repos
https://api.github.com/users/EduardoPach/starred{/owner}{/repo}
https://api.github.com/users/EduardoPach/subscriptions
User
https://api.github.com/users/EduardoPach
public
COLLABORATOR
@EduardoPach This issue is still relevant, so feel free to work on it.
2023-07-26T16:59:30Z
https://github.com/huggingface/datasets/issues/5806#issuecomment-1652186892
https://api.github.com/repos/huggingface/datasets/issues/5806
IC_kwDODunzps5iel8M
https://api.github.com/repos/huggingface/datasets/issues/comments/1652186892/reactions
2023-07-26T16:59:30Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1652186892
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
NONE
Hey @mariosasko, I've taken the time to take a look at how we load the datasets usually. My main question now is about the final solution.

So the idea is that whenever we load the datasets we also add a new column in the _generate_tables() method from the builders called filename (or file_name) that should be related files contained in each split, right?

Do you have any suggestions of where I could add that? 
2023-07-28T22:08:17Z
https://github.com/huggingface/datasets/issues/5806#issuecomment-1656379053
https://api.github.com/repos/huggingface/datasets/issues/5806
IC_kwDODunzps5iulat
https://api.github.com/repos/huggingface/datasets/issues/comments/1656379053/reactions
2023-07-28T22:08:17Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1656379053
https://avatars.githubusercontent.com/u/69953243?v=4
https://api.github.com/users/EduardoPach/events{/privacy}
https://api.github.com/users/EduardoPach/followers
https://api.github.com/users/EduardoPach/following{/other_user}
https://api.github.com/users/EduardoPach/gists{/gist_id}
https://github.com/EduardoPach
EduardoPach
MDQ6VXNlcjY5OTUzMjQz
https://api.github.com/users/EduardoPach/orgs
https://api.github.com/users/EduardoPach/received_events
https://api.github.com/users/EduardoPach/repos
https://api.github.com/users/EduardoPach/starred{/owner}{/repo}
https://api.github.com/users/EduardoPach/subscriptions
User
https://api.github.com/users/EduardoPach
public
NONE
Is this issue still open? If yes, I'd like to work upon on it. Thanks
2023-09-28T18:15:54Z
https://github.com/huggingface/datasets/issues/5806#issuecomment-1739800155
https://api.github.com/repos/huggingface/datasets/issues/5806
IC_kwDODunzps5nsz5b
https://api.github.com/repos/huggingface/datasets/issues/comments/1739800155/reactions
2023-09-28T18:15:54Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1739800155
https://avatars.githubusercontent.com/u/76659791?v=4
https://api.github.com/users/BattiniSandeep/events{/privacy}
https://api.github.com/users/BattiniSandeep/followers
https://api.github.com/users/BattiniSandeep/following{/other_user}
https://api.github.com/users/BattiniSandeep/gists{/gist_id}
https://github.com/BattiniSandeep
BattiniSandeep
MDQ6VXNlcjc2NjU5Nzkx
https://api.github.com/users/BattiniSandeep/orgs
https://api.github.com/users/BattiniSandeep/received_events
https://api.github.com/users/BattiniSandeep/repos
https://api.github.com/users/BattiniSandeep/starred{/owner}{/repo}
https://api.github.com/users/BattiniSandeep/subscriptions
User
https://api.github.com/users/BattiniSandeep
public
NONE
> Is this issue still open? If yes, I'd like to work upon on it. Thanks

Definitely still open. I gave it a try, but then didn't get any feedback on my last question so I stopped . Feel free to work on it.
2023-09-28T18:53:31Z
https://github.com/huggingface/datasets/issues/5806#issuecomment-1739847408
https://api.github.com/repos/huggingface/datasets/issues/5806
IC_kwDODunzps5ns_bw
https://api.github.com/repos/huggingface/datasets/issues/comments/1739847408/reactions
2023-09-28T18:54:10Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1739847408
https://avatars.githubusercontent.com/u/69953243?v=4
https://api.github.com/users/EduardoPach/events{/privacy}
https://api.github.com/users/EduardoPach/followers
https://api.github.com/users/EduardoPach/following{/other_user}
https://api.github.com/users/EduardoPach/gists{/gist_id}
https://github.com/EduardoPach
EduardoPach
MDQ6VXNlcjY5OTUzMjQz
https://api.github.com/users/EduardoPach/orgs
https://api.github.com/users/EduardoPach/received_events
https://api.github.com/users/EduardoPach/repos
https://api.github.com/users/EduardoPach/starred{/owner}{/repo}
https://api.github.com/users/EduardoPach/subscriptions
User
https://api.github.com/users/EduardoPach
public
COLLABORATOR
It's still open, so feel free to work on it. This can be implemented by adding a param to the packaged builders' configs that inserts a column with file names (in `_generate_tables`) when `True`. Naming this column `file_name` sounds good to me.
2023-09-29T17:49:53Z
https://github.com/huggingface/datasets/issues/5806#issuecomment-1741279266
https://api.github.com/repos/huggingface/datasets/issues/5806
IC_kwDODunzps5nydAi
https://api.github.com/repos/huggingface/datasets/issues/comments/1741279266/reactions
2023-09-29T17:49:53Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1741279266
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
NONE
Hi is the issues still open, is see no activity since September but it shows that it is still assigned to tsabbir96.  If 
tsabbir96 is not planning to continue, can i get it assigned to me.
2024-01-21T16:38:28Z
https://github.com/huggingface/datasets/issues/5806#issuecomment-1902689900
https://api.github.com/repos/huggingface/datasets/issues/5806
IC_kwDODunzps5xaL5s
https://api.github.com/repos/huggingface/datasets/issues/comments/1902689900/reactions
2024-01-21T16:38:28Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1902689900
https://avatars.githubusercontent.com/u/47289626?v=4
https://api.github.com/users/aniruddh-23/events{/privacy}
https://api.github.com/users/aniruddh-23/followers
https://api.github.com/users/aniruddh-23/following{/other_user}
https://api.github.com/users/aniruddh-23/gists{/gist_id}
https://github.com/aniruddh-23
aniruddh-23
MDQ6VXNlcjQ3Mjg5NjI2
https://api.github.com/users/aniruddh-23/orgs
https://api.github.com/users/aniruddh-23/received_events
https://api.github.com/users/aniruddh-23/repos
https://api.github.com/users/aniruddh-23/starred{/owner}{/repo}
https://api.github.com/users/aniruddh-23/subscriptions
User
https://api.github.com/users/aniruddh-23
public
NONE
Looking forward to your implementation. I also really need this feature. 
Thanks
2024-04-12T01:37:51Z
https://github.com/huggingface/datasets/issues/5806#issuecomment-2050813372
https://api.github.com/repos/huggingface/datasets/issues/5806
IC_kwDODunzps56PO28
https://api.github.com/repos/huggingface/datasets/issues/comments/2050813372/reactions
2024-04-12T01:37:51Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2050813372
https://avatars.githubusercontent.com/u/28639377?v=4
https://api.github.com/users/SWHL/events{/privacy}
https://api.github.com/users/SWHL/followers
https://api.github.com/users/SWHL/following{/other_user}
https://api.github.com/users/SWHL/gists{/gist_id}
https://github.com/SWHL
SWHL
MDQ6VXNlcjI4NjM5Mzc3
https://api.github.com/users/SWHL/orgs
https://api.github.com/users/SWHL/received_events
https://api.github.com/users/SWHL/repos
https://api.github.com/users/SWHL/starred{/owner}{/repo}
https://api.github.com/users/SWHL/subscriptions
User
https://api.github.com/users/SWHL
public
NONE
Hi. I am new and would like to contribute to this issue @tsabbir96
2024-10-02T13:11:49Z
https://github.com/huggingface/datasets/issues/5806#issuecomment-2388610536
https://api.github.com/repos/huggingface/datasets/issues/5806
IC_kwDODunzps6OX03o
https://api.github.com/repos/huggingface/datasets/issues/comments/2388610536/reactions
2024-10-02T13:11:49Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2388610536
https://avatars.githubusercontent.com/u/72044811?v=4
https://api.github.com/users/DhanushA1307/events{/privacy}
https://api.github.com/users/DhanushA1307/followers
https://api.github.com/users/DhanushA1307/following{/other_user}
https://api.github.com/users/DhanushA1307/gists{/gist_id}
https://github.com/DhanushA1307
DhanushA1307
MDQ6VXNlcjcyMDQ0ODEx
https://api.github.com/users/DhanushA1307/orgs
https://api.github.com/users/DhanushA1307/received_events
https://api.github.com/users/DhanushA1307/repos
https://api.github.com/users/DhanushA1307/starred{/owner}{/repo}
https://api.github.com/users/DhanushA1307/subscriptions
User
https://api.github.com/users/DhanushA1307
public
NONE
Hi,is this issue still open?if yes ,I d like to work on it .Thanks
2025-03-21T12:07:13Z
https://github.com/huggingface/datasets/issues/5806#issuecomment-2743181764
https://api.github.com/repos/huggingface/datasets/issues/5806
IC_kwDODunzps6jgaHE
https://api.github.com/repos/huggingface/datasets/issues/comments/2743181764/reactions
2025-03-21T12:07:13Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2743181764
https://avatars.githubusercontent.com/u/114857649?v=4
https://api.github.com/users/Myriamdamak/events{/privacy}
https://api.github.com/users/Myriamdamak/followers
https://api.github.com/users/Myriamdamak/following{/other_user}
https://api.github.com/users/Myriamdamak/gists{/gist_id}
https://github.com/Myriamdamak
Myriamdamak
U_kgDOBtiWsQ
https://api.github.com/users/Myriamdamak/orgs
https://api.github.com/users/Myriamdamak/received_events
https://api.github.com/users/Myriamdamak/repos
https://api.github.com/users/Myriamdamak/starred{/owner}{/repo}
https://api.github.com/users/Myriamdamak/subscriptions
User
https://api.github.com/users/Myriamdamak
public
### Feature request

Add an optional parameter return_file_name in the load_dataset function. When it is set to True, the function will include the name of the file corresponding to the current line as a feature in the returned output.

### Motivation

When training large language models, machine problems may interrupt the training process. In such cases, it is common to load a previously saved checkpoint to resume training. I would like to be able to obtain the names of the previously trained data shards, so that I can skip these parts of the data during continued training to avoid overfitting and redundant training time.

### Your contribution

I currently use a dataset in jsonl format, so I am primarily interested in the json format. I suggest adding the file name to the returned table here https://github.com/huggingface/datasets/blob/main/src/datasets/packaged_modules/json/json.py#L92.