How to convert  torch.utils.data.Dataset to huggingface dataset?
COLLABORATOR
Hi! I think you can use the newly-added `from_generator` method for that:
```python
from datasets import Dataset

def gen():
    for idx in len(torch_dataset):
        yield torch_dataset[idx]  # this has to be a dictionary
    ## or if it's an IterableDataset
    # for ex in torch_dataset:
    #     yield ex

dset = Dataset.from_generator(gen)
```
2022-09-16T15:28:26Z
https://github.com/huggingface/datasets/issues/4983#issuecomment-1249504992
https://api.github.com/repos/huggingface/datasets/issues/4983
IC_kwDODunzps5Kee7g
https://api.github.com/repos/huggingface/datasets/issues/comments/1249504992/reactions
2022-09-16T15:28:26Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1249504992
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
MEMBER
Maybe `Dataset.from_list` can work as well no ?
```python
from datasets import Dataset

dset = Dataset.from_list(torch_dataset)
```
2022-09-19T14:47:18Z
https://github.com/huggingface/datasets/issues/4983#issuecomment-1251127025
https://api.github.com/repos/huggingface/datasets/issues/4983
IC_kwDODunzps5Kkq7x
https://api.github.com/repos/huggingface/datasets/issues/comments/1251127025/reactions
2022-09-19T14:47:18Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1251127025
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
> ```python
> from datasets import Dataset
> 
> def gen():
>     for idx in len(torch_dataset):
>         yield torch_dataset[idx]  # this has to be a dictionary
>     ## or if it's an IterableDataset
>     # for ex in torch_dataset:
>     #     yield ex
> 
> dset = Dataset.from_generator(gen)
> ```

I try to use `Dataset.from_generator()` method, and it returns an error:
```bash
AttributeError: type object 'Dataset' has no attribute 'from_generator'
```
And I think it maybe the version of my datasets package is out-of-date, so I update it
```bash
pip install --upgrade datasets
```
But after that, the code still return the above Error. 
2022-09-20T01:57:58Z
https://github.com/huggingface/datasets/issues/4983#issuecomment-1251744864
https://api.github.com/repos/huggingface/datasets/issues/4983
IC_kwDODunzps5KnBxg
https://api.github.com/repos/huggingface/datasets/issues/comments/1251744864/reactions
2022-09-20T01:57:58Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1251744864
https://avatars.githubusercontent.com/u/77595952?v=4
https://api.github.com/users/DEROOCE/events{/privacy}
https://api.github.com/users/DEROOCE/followers
https://api.github.com/users/DEROOCE/following{/other_user}
https://api.github.com/users/DEROOCE/gists{/gist_id}
https://github.com/DEROOCE
DEROOCE
MDQ6VXNlcjc3NTk1OTUy
https://api.github.com/users/DEROOCE/orgs
https://api.github.com/users/DEROOCE/received_events
https://api.github.com/users/DEROOCE/repos
https://api.github.com/users/DEROOCE/starred{/owner}{/repo}
https://api.github.com/users/DEROOCE/subscriptions
User
https://api.github.com/users/DEROOCE
public
NONE
> ```python
> dset = Dataset.from_list(torch_dataset)
> ```

It seems that Dataset also has no `from_list` method 😂
```bash
AttributeError: type object 'Dataset' has no attribute 'from_list'
```
2022-09-20T02:04:44Z
https://github.com/huggingface/datasets/issues/4983#issuecomment-1251748069
https://api.github.com/repos/huggingface/datasets/issues/4983
IC_kwDODunzps5KnCjl
https://api.github.com/repos/huggingface/datasets/issues/comments/1251748069/reactions
2022-09-20T02:04:44Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1251748069
https://avatars.githubusercontent.com/u/77595952?v=4
https://api.github.com/users/DEROOCE/events{/privacy}
https://api.github.com/users/DEROOCE/followers
https://api.github.com/users/DEROOCE/following{/other_user}
https://api.github.com/users/DEROOCE/gists{/gist_id}
https://github.com/DEROOCE
DEROOCE
MDQ6VXNlcjc3NTk1OTUy
https://api.github.com/users/DEROOCE/orgs
https://api.github.com/users/DEROOCE/received_events
https://api.github.com/users/DEROOCE/repos
https://api.github.com/users/DEROOCE/starred{/owner}{/repo}
https://api.github.com/users/DEROOCE/subscriptions
User
https://api.github.com/users/DEROOCE
public
NONE
> I look through the huggingface dataset docs, and it seems that there is no offical support function to convert `torch.utils.data.Dataset` to huggingface dataset. However, there is a way to convert huggingface dataset to `torch.utils.data.Dataset`, like below:
> 
> ```python
> from datasets import Dataset
> data = [[1, 2],[3, 4]]
> ds = Dataset.from_dict({"data": data})
> ds = ds.with_format("torch")
> ds[0]
> ds[:2]
> ```
> 
> So is there something I miss, or there IS no function to convert `torch.utils.data.Dataset` to huggingface dataset. If so, is there any way to do this convert? Thanks.

My dummy code is like:
```python
import os
import json
from torch.utils import data
import datasets

def gen(torch_dataset):
    for idx in len(torch_dataset):
        yield torch_dataset[idx]  # this has to be a dictionary

class MyDataset(data.Dataset):
    def __init__(self, path):
        self.dict = []
        for line in open(path, 'r', encoding='utf-8'):
            j_dict = json.loads(line)
            self.dict.append(j_dict['context'])
            
    def __getitem__(self, idx):
        return self.dict[idx]

    def __len__(self):
        return len(self.dict)

root_path = os.path.dirname(os.path.abspath(__file__))
path = os.path.join(root_path, 'dataset', 'train.json')
torch_dataset = MyDataset(path)

dit = []
for line in open(path, 'r', encoding='utf-8'):
    j_dict = json.loads(line)
    dit.append(j_dict['context'])
dset1 = datasets.Dataset.from_list(dit)
print(dset1)
dset2 = datasets.Dataset.from_generator(gen)
print(dset2)
```
2022-09-20T02:05:07Z
https://github.com/huggingface/datasets/issues/4983#issuecomment-1251748248
https://api.github.com/repos/huggingface/datasets/issues/4983
IC_kwDODunzps5KnCmY
https://api.github.com/repos/huggingface/datasets/issues/comments/1251748248/reactions
2022-09-20T02:05:07Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1251748248
https://avatars.githubusercontent.com/u/77595952?v=4
https://api.github.com/users/DEROOCE/events{/privacy}
https://api.github.com/users/DEROOCE/followers
https://api.github.com/users/DEROOCE/following{/other_user}
https://api.github.com/users/DEROOCE/gists{/gist_id}
https://github.com/DEROOCE
DEROOCE
MDQ6VXNlcjc3NTk1OTUy
https://api.github.com/users/DEROOCE/orgs
https://api.github.com/users/DEROOCE/received_events
https://api.github.com/users/DEROOCE/repos
https://api.github.com/users/DEROOCE/starred{/owner}{/repo}
https://api.github.com/users/DEROOCE/subscriptions
User
https://api.github.com/users/DEROOCE
public
MEMBER
We're releasing `from_generator` and `from_list` today :)
In the meantime you can play with them by installing `datasets` from source
2022-09-20T09:46:04Z
https://github.com/huggingface/datasets/issues/4983#issuecomment-1252106585
https://api.github.com/repos/huggingface/datasets/issues/4983
IC_kwDODunzps5KoaFZ
https://api.github.com/repos/huggingface/datasets/issues/comments/1252106585/reactions
2022-09-20T09:46:04Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1252106585
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
> We're releasing `from_generator` and `from_list` today :) In the meantime you can play with them by installing `datasets` from source

Thanks a lot for your work!
2022-09-20T11:23:38Z
https://github.com/huggingface/datasets/issues/4983#issuecomment-1252211666
https://api.github.com/repos/huggingface/datasets/issues/4983
IC_kwDODunzps5KozvS
https://api.github.com/repos/huggingface/datasets/issues/comments/1252211666/reactions
2022-09-20T11:23:38Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1252211666
https://avatars.githubusercontent.com/u/77595952?v=4
https://api.github.com/users/DEROOCE/events{/privacy}
https://api.github.com/users/DEROOCE/followers
https://api.github.com/users/DEROOCE/following{/other_user}
https://api.github.com/users/DEROOCE/gists{/gist_id}
https://github.com/DEROOCE
DEROOCE
MDQ6VXNlcjc3NTk1OTUy
https://api.github.com/users/DEROOCE/orgs
https://api.github.com/users/DEROOCE/received_events
https://api.github.com/users/DEROOCE/repos
https://api.github.com/users/DEROOCE/starred{/owner}{/repo}
https://api.github.com/users/DEROOCE/subscriptions
User
https://api.github.com/users/DEROOCE
public
NONE
> > I look through the huggingface dataset docs, and it seems that there is no offical support function to convert `torch.utils.data.Dataset` to huggingface dataset. However, there is a way to convert huggingface dataset to `torch.utils.data.Dataset`, like below:
> > ```python
> > from datasets import Dataset
> > data = [[1, 2],[3, 4]]
> > ds = Dataset.from_dict({"data": data})
> > ds = ds.with_format("torch")
> > ds[0]
> > ds[:2]
> > ```
> > 
> > 
> >     
> >       
> >     
> > 
> >       
> >     
> > 
> >     
> >   
> > So is there something I miss, or there IS no function to convert `torch.utils.data.Dataset` to huggingface dataset. If so, is there any way to do this convert? Thanks.
> 
> My dummy code is like:
> 
> ```python
> import os
> import json
> from torch.utils import data
> import datasets
> 
> def gen(torch_dataset):
>     for idx in len(torch_dataset):
>         yield torch_dataset[idx]  # this has to be a dictionary
> 
> class MyDataset(data.Dataset):
>     def __init__(self, path):
>         self.dict = []
>         for line in open(path, 'r', encoding='utf-8'):
>             j_dict = json.loads(line)
>             self.dict.append(j_dict['context'])
>             
>     def __getitem__(self, idx):
>         return self.dict[idx]
> 
>     def __len__(self):
>         return len(self.dict)
> 
> root_path = os.path.dirname(os.path.abspath(__file__))
> path = os.path.join(root_path, 'dataset', 'train.json')
> torch_dataset = MyDataset(path)
> 
> dit = []
> for line in open(path, 'r', encoding='utf-8'):
>     j_dict = json.loads(line)
>     dit.append(j_dict['context'])
> dset1 = datasets.Dataset.from_list(dit)
> print(dset1)
> dset2 = datasets.Dataset.from_generator(gen)
> print(dset2)
> ```
Hi, when I am using this code to build my own dataset, ` datasets.Dataset.from_generator(gen)` report `TypeError: cannot pickle generator object` whre MyDataset returns a dict like {'image': bytes, 'text': string}. How can I resolve this? Thanks a lot!
2023-03-30T11:12:15Z
https://github.com/huggingface/datasets/issues/4983#issuecomment-1490118622
https://api.github.com/repos/huggingface/datasets/issues/4983
IC_kwDODunzps5Y0Wfe
https://api.github.com/repos/huggingface/datasets/issues/comments/1490118622/reactions
2023-03-30T11:12:15Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1490118622
https://avatars.githubusercontent.com/u/9052195?v=4
https://api.github.com/users/winnechan/events{/privacy}
https://api.github.com/users/winnechan/followers
https://api.github.com/users/winnechan/following{/other_user}
https://api.github.com/users/winnechan/gists{/gist_id}
https://github.com/winnechan
winnechan
MDQ6VXNlcjkwNTIxOTU=
https://api.github.com/users/winnechan/orgs
https://api.github.com/users/winnechan/received_events
https://api.github.com/users/winnechan/repos
https://api.github.com/users/winnechan/starred{/owner}{/repo}
https://api.github.com/users/winnechan/subscriptions
User
https://api.github.com/users/winnechan
public
MEMBER
Hi ! Right now generator functions are expected to be picklable, so that `datasets` can hash it and use the hash to cache the resulting Dataset on disk. Maybe this can be improved.

In the meantime, can you check that you're not using unpickable objects. In your case it looks like you're using a generator object that is unpickable. It might come from an opened file, e.g. this doesn't work:
```python
with open(...) as f:

    def gen():
        for x in f:
            yield json.loads(x)

    ds = Dataset.from_generator(gen)
```
but this does work:
```python
def gen():
    with open(...) as f:
        for x in f:
            yield json.loads(x)

ds = Dataset.from_generator(gen)
```
2023-03-30T12:42:36Z
https://github.com/huggingface/datasets/issues/4983#issuecomment-1490236400
https://api.github.com/repos/huggingface/datasets/issues/4983
IC_kwDODunzps5Y0zPw
https://api.github.com/repos/huggingface/datasets/issues/comments/1490236400/reactions
2023-03-30T12:42:36Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1490236400
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
> Hi ! Right now generator functions are expected to be picklable, so that `datasets` can hash it and use the hash to cache the resulting Dataset on disk. Maybe this can be improved.
> 
> In the meantime, can you check that you're not using unpickable objects. In your case it looks like you're using a generator object that is unpickable. It might come from an opened file, e.g. this doesn't work:
> 
> ```python
> with open(...) as f:
> 
>     def gen():
>         for x in f:
>             yield json.loads(x)
> 
>     ds = Dataset.from_generator(gen)
> ```
> 
> but this does work:
> 
> ```python
> def gen():
>     with open(...) as f:
>         for x in f:
>             yield json.loads(x)
> 
> ds = Dataset.from_generator(gen)
> ```

Thanks a lot! That's the reason why I have encountered this issue. Sorry for bothering you again with another problem, since my dataset is large and I use IterableDataset.from_generator which has no attribute with_transform, how can I equip it with some customed preprocessings like Dataset.from_generator? Should I move the preprocessing to the my torch Dataset?
2023-03-30T12:55:37Z
https://github.com/huggingface/datasets/issues/4983#issuecomment-1490256610
https://api.github.com/repos/huggingface/datasets/issues/4983
IC_kwDODunzps5Y04Li
https://api.github.com/repos/huggingface/datasets/issues/comments/1490256610/reactions
2023-03-30T12:55:37Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1490256610
https://avatars.githubusercontent.com/u/9052195?v=4
https://api.github.com/users/winnechan/events{/privacy}
https://api.github.com/users/winnechan/followers
https://api.github.com/users/winnechan/following{/other_user}
https://api.github.com/users/winnechan/gists{/gist_id}
https://github.com/winnechan
winnechan
MDQ6VXNlcjkwNTIxOTU=
https://api.github.com/users/winnechan/orgs
https://api.github.com/users/winnechan/received_events
https://api.github.com/users/winnechan/repos
https://api.github.com/users/winnechan/starred{/owner}{/repo}
https://api.github.com/users/winnechan/subscriptions
User
https://api.github.com/users/winnechan
public
MEMBER
Iterable datasets are lazy: exactly like `with_transform` they apply processing on the fly when accessing the examples.

Therefore you can use `my_iterable_dataset.map()` instead :)
2023-03-30T14:51:09Z
https://github.com/huggingface/datasets/issues/4983#issuecomment-1490444711
https://api.github.com/repos/huggingface/datasets/issues/4983
IC_kwDODunzps5Y1mGn
https://api.github.com/repos/huggingface/datasets/issues/comments/1490444711/reactions
2023-03-30T14:51:09Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1490444711
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
@lhoestq thanks a lot and I have successfully made it work~
2023-03-31T06:31:41Z
https://github.com/huggingface/datasets/issues/4983#issuecomment-1491385104
https://api.github.com/repos/huggingface/datasets/issues/4983
IC_kwDODunzps5Y5LsQ
https://api.github.com/repos/huggingface/datasets/issues/comments/1491385104/reactions
2023-03-31T06:31:41Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1491385104
https://avatars.githubusercontent.com/u/9052195?v=4
https://api.github.com/users/winnechan/events{/privacy}
https://api.github.com/users/winnechan/followers
https://api.github.com/users/winnechan/following{/other_user}
https://api.github.com/users/winnechan/gists{/gist_id}
https://github.com/winnechan
winnechan
MDQ6VXNlcjkwNTIxOTU=
https://api.github.com/users/winnechan/orgs
https://api.github.com/users/winnechan/received_events
https://api.github.com/users/winnechan/repos
https://api.github.com/users/winnechan/starred{/owner}{/repo}
https://api.github.com/users/winnechan/subscriptions
User
https://api.github.com/users/winnechan
public
NONE
@lhoestq  I am having a similar issue. Can you help me understand which kinds of generators are picklable? I previously thought that no generators are picklable so I'm intrigued to hear this.
2023-05-02T15:36:08Z
https://github.com/huggingface/datasets/issues/4983#issuecomment-1531690046
https://api.github.com/repos/huggingface/datasets/issues/4983
IC_kwDODunzps5bS7w-
https://api.github.com/repos/huggingface/datasets/issues/comments/1531690046/reactions
2023-05-02T15:36:40Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1531690046
https://avatars.githubusercontent.com/u/19317207?v=4
https://api.github.com/users/atyshka/events{/privacy}
https://api.github.com/users/atyshka/followers
https://api.github.com/users/atyshka/following{/other_user}
https://api.github.com/users/atyshka/gists{/gist_id}
https://github.com/atyshka
atyshka
MDQ6VXNlcjE5MzE3MjA3
https://api.github.com/users/atyshka/orgs
https://api.github.com/users/atyshka/received_events
https://api.github.com/users/atyshka/repos
https://api.github.com/users/atyshka/starred{/owner}{/repo}
https://api.github.com/users/atyshka/subscriptions
User
https://api.github.com/users/atyshka
public
MEMBER
Generator functions are generally picklable. E.g.
```python
import dill as pickle

def generator_fn():
    for i in range(10):
        yield i

pickle.dumps(generator_fn)
```

however generators are not picklable
```python
generator = generator_fn()
pickle.dumps(generator)
# TypeError: cannot pickle 'generator' object
```

Though it can happen that some generator functions are not recursively picklable if they use global objects that are not picklable:
```python
def generator_fn_not_picklable():
    for i in generator:
        yield i

pickle.dumps(generator_fn_not_picklable, recurse=True)
# TypeError: cannot pickle 'generator' object
````
2023-05-05T14:20:06Z
https://github.com/huggingface/datasets/issues/4983#issuecomment-1536335657
https://api.github.com/repos/huggingface/datasets/issues/4983
IC_kwDODunzps5bkp8p
https://api.github.com/repos/huggingface/datasets/issues/comments/1536335657/reactions
2023-05-05T14:20:06Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1536335657
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
I'm trying to create an IterableDataset from a generator but I get this error:
`PicklingError: Can't pickle <built-in function input>: it's not the same object as builtins.input`

What can I do?
2023-12-14T20:54:15Z
https://github.com/huggingface/datasets/issues/4983#issuecomment-1856573841
https://api.github.com/repos/huggingface/datasets/issues/4983
IC_kwDODunzps5uqRGR
https://api.github.com/repos/huggingface/datasets/issues/comments/1856573841/reactions
2023-12-14T20:54:15Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1856573841
https://avatars.githubusercontent.com/u/44657733?v=4
https://api.github.com/users/AeroDEmi/events{/privacy}
https://api.github.com/users/AeroDEmi/followers
https://api.github.com/users/AeroDEmi/following{/other_user}
https://api.github.com/users/AeroDEmi/gists{/gist_id}
https://github.com/AeroDEmi
AeroDEmi
MDQ6VXNlcjQ0NjU3NzMz
https://api.github.com/users/AeroDEmi/orgs
https://api.github.com/users/AeroDEmi/received_events
https://api.github.com/users/AeroDEmi/repos
https://api.github.com/users/AeroDEmi/starred{/owner}{/repo}
https://api.github.com/users/AeroDEmi/subscriptions
User
https://api.github.com/users/AeroDEmi
public
I look through the huggingface dataset docs, and it seems that there is no offical support function to  convert  `torch.utils.data.Dataset`  to huggingface dataset. However, there is a way to convert huggingface dataset to  `torch.utils.data.Dataset`, like below:
```python
from datasets import Dataset
data = [[1, 2],[3, 4]]
ds = Dataset.from_dict({"data": data})
ds = ds.with_format("torch")
ds[0]
ds[:2]
```
So is there something I miss, or there IS no function to convert  `torch.utils.data.Dataset` to huggingface dataset. If so, is there any way to do this convert?
Thanks.