new dataset type: single-label and multi-label video classification
MEMBER
Hi ! You can in the `features` folder how we implemented the audio and image feature types.

We can have something similar to videos. What we need to decide:
- the video loading library to use
- the output format when a user accesses a video type object
- what parameters a `Video()` feature type needs

also cc @nateraw who also took a look at what we can do for video
2022-09-30T15:29:27Z
https://github.com/huggingface/datasets/issues/5032#issuecomment-1263719571
https://api.github.com/repos/huggingface/datasets/issues/5032
IC_kwDODunzps5LUtST
https://api.github.com/repos/huggingface/datasets/issues/comments/1263719571/reactions
2022-09-30T15:29:27Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1263719571
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
@lhoestq @nateraw is there any progress on adding video classification datasets? 
2022-11-02T07:47:19Z
https://github.com/huggingface/datasets/issues/5032#issuecomment-1299711808
https://api.github.com/repos/huggingface/datasets/issues/5032
IC_kwDODunzps5NeAdA
https://api.github.com/repos/huggingface/datasets/issues/comments/1299711808/reactions
2022-11-02T07:47:19Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1299711808
https://avatars.githubusercontent.com/u/22957388?v=4
https://api.github.com/users/sayakpaul/events{/privacy}
https://api.github.com/users/sayakpaul/followers
https://api.github.com/users/sayakpaul/following{/other_user}
https://api.github.com/users/sayakpaul/gists{/gist_id}
https://github.com/sayakpaul
sayakpaul
MDQ6VXNlcjIyOTU3Mzg4
https://api.github.com/users/sayakpaul/orgs
https://api.github.com/users/sayakpaul/received_events
https://api.github.com/users/sayakpaul/repos
https://api.github.com/users/sayakpaul/starred{/owner}{/repo}
https://api.github.com/users/sayakpaul/subscriptions
User
https://api.github.com/users/sayakpaul
public
MEMBER
Hi ! I think we just missing which lib we're going to use to decode the videos + which parameters must go in the `Video` type
2022-11-02T10:19:18Z
https://github.com/huggingface/datasets/issues/5032#issuecomment-1299992167
https://api.github.com/repos/huggingface/datasets/issues/5032
IC_kwDODunzps5NfE5n
https://api.github.com/repos/huggingface/datasets/issues/comments/1299992167/reactions
2022-11-02T10:19:18Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1299992167
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
Hmm. `decord` could be nice but it's no longer maintained [it seems](https://github.com/dmlc/decord/issues/214). 
2022-11-02T11:57:11Z
https://github.com/huggingface/datasets/issues/5032#issuecomment-1300213925
https://api.github.com/repos/huggingface/datasets/issues/5032
IC_kwDODunzps5Nf7Cl
https://api.github.com/repos/huggingface/datasets/issues/comments/1300213925/reactions
2022-11-02T11:57:21Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1300213925
https://avatars.githubusercontent.com/u/22957388?v=4
https://api.github.com/users/sayakpaul/events{/privacy}
https://api.github.com/users/sayakpaul/followers
https://api.github.com/users/sayakpaul/following{/other_user}
https://api.github.com/users/sayakpaul/gists{/gist_id}
https://github.com/sayakpaul
sayakpaul
MDQ6VXNlcjIyOTU3Mzg4
https://api.github.com/users/sayakpaul/orgs
https://api.github.com/users/sayakpaul/received_events
https://api.github.com/users/sayakpaul/repos
https://api.github.com/users/sayakpaul/starred{/owner}{/repo}
https://api.github.com/users/sayakpaul/subscriptions
User
https://api.github.com/users/sayakpaul
public
NONE
pytorchvideo uses [pyav](https://github.com/PyAV-Org/PyAV) as the default decoder: https://github.com/facebookresearch/pytorchvideo/blob/c8d23d8b7e597586a9e2d18f6ed31ad8aa379a7a/pytorchvideo/data/labeled_video_dataset.py#L37

Also it would be great if `optionally` audio can also be decoded from the video as in pytorchvideo: https://github.com/facebookresearch/pytorchvideo/blob/c8d23d8b7e597586a9e2d18f6ed31ad8aa379a7a/pytorchvideo/data/labeled_video_dataset.py#L35

Here are the other decoders supported in pytorchvideo: https://github.com/facebookresearch/pytorchvideo/blob/c8d23d8b7e597586a9e2d18f6ed31ad8aa379a7a/pytorchvideo/data/encoded_video.py#L17

2022-11-02T13:11:51Z
https://github.com/huggingface/datasets/issues/5032#issuecomment-1300386297
https://api.github.com/repos/huggingface/datasets/issues/5032
IC_kwDODunzps5NglH5
https://api.github.com/repos/huggingface/datasets/issues/comments/1300386297/reactions
2022-11-02T13:19:15Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1300386297
https://avatars.githubusercontent.com/u/34196005?v=4
https://api.github.com/users/fcakyon/events{/privacy}
https://api.github.com/users/fcakyon/followers
https://api.github.com/users/fcakyon/following{/other_user}
https://api.github.com/users/fcakyon/gists{/gist_id}
https://github.com/fcakyon
fcakyon
MDQ6VXNlcjM0MTk2MDA1
https://api.github.com/users/fcakyon/orgs
https://api.github.com/users/fcakyon/received_events
https://api.github.com/users/fcakyon/repos
https://api.github.com/users/fcakyon/starred{/owner}{/repo}
https://api.github.com/users/fcakyon/subscriptions
User
https://api.github.com/users/fcakyon
public
CONTRIBUTOR
@sayakpaul I did do quite a bit of work on [this PR](https://github.com/huggingface/datasets/pull/4532) a while back to add a video feature. It's outdated, but uses my `encoded_video` [package](https://github.com/nateraw/encoded-video) under the hood, which is basically a wrapper around PyAV stolen from [pytorchvideo](https://github.com/facebookresearch/pytorchvideo/) that gets rid of the `torch` dependency. 

would be really great to get something like this in...it's just a really tricky and time consuming feature to add. 
2022-11-02T19:10:13Z
https://github.com/huggingface/datasets/issues/5032#issuecomment-1301095733
https://api.github.com/repos/huggingface/datasets/issues/5032
IC_kwDODunzps5NjSU1
https://api.github.com/repos/huggingface/datasets/issues/comments/1301095733/reactions
2022-11-02T19:10:13Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1301095733
https://avatars.githubusercontent.com/u/32437151?v=4
https://api.github.com/users/nateraw/events{/privacy}
https://api.github.com/users/nateraw/followers
https://api.github.com/users/nateraw/following{/other_user}
https://api.github.com/users/nateraw/gists{/gist_id}
https://github.com/nateraw
nateraw
MDQ6VXNlcjMyNDM3MTUx
https://api.github.com/users/nateraw/orgs
https://api.github.com/users/nateraw/received_events
https://api.github.com/users/nateraw/repos
https://api.github.com/users/nateraw/starred{/owner}{/repo}
https://api.github.com/users/nateraw/subscriptions
User
https://api.github.com/users/nateraw
public
**Is your feature request related to a problem? Please describe.**
In my research, I am dealing with multi-modal (audio+text+frame sequence) video classification. It would be great if the datasets library supported generating multi-modal batches from a video dataset.

**Describe the solution you'd like**
Assume I have video files having single/multiple labels. I want to train a single/multi-label video classification model. I want datasets to support generating multi-modal batches (audio+frame sequence) from video files. Audio waveform and frame sequence can be extracted from each video clip then I can use any audio, image and video model from transformers library to extract features which will be fed into my model.

**Describe alternatives you've considered**
Currently, I am using https://github.com/facebookresearch/pytorchvideo dataloaders. There seems to be not much alternative.

**Additional context**
I am wiling to open a PR but don't know where to start.
