How to train BERT model with next sentence prediction?
MEMBER
Hi !
We're not using `TextDatasetForNextSentencePrediction` in `datasets`.
Although you can probably use the `TextDatasetForNextSentencePrediction.create_examples_from_document` on a dataset to prepare it for next sentence prediction.
2021-03-29T13:34:04Z
https://github.com/huggingface/datasets/issues/2129#issuecomment-809381124
https://api.github.com/repos/huggingface/datasets/issues/2129
MDEyOklzc3VlQ29tbWVudDgwOTM4MTEyNA==
https://api.github.com/repos/huggingface/datasets/issues/comments/809381124/reactions
2021-03-29T13:34:04Z
https://api.github.com/repos/huggingface/datasets/issues/comments/809381124
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
Thanks.

Do you mean that `TextDatasetForNextSentencePrediction.create_exapmles_from_document` can be applied to dataset object other than `TextDatasetForNextSentencePrediction`  e.g. a `Dataset` object which is  loaded by  `datasets.load_dataset`?
2021-03-29T14:20:39Z
https://github.com/huggingface/datasets/issues/2129#issuecomment-809418864
https://api.github.com/repos/huggingface/datasets/issues/2129
MDEyOklzc3VlQ29tbWVudDgwOTQxODg2NA==
https://api.github.com/repos/huggingface/datasets/issues/comments/809418864/reactions
2021-03-29T14:20:39Z
https://api.github.com/repos/huggingface/datasets/issues/comments/809418864
https://avatars.githubusercontent.com/u/836541?v=4
https://api.github.com/users/jnishi/events{/privacy}
https://api.github.com/users/jnishi/followers
https://api.github.com/users/jnishi/following{/other_user}
https://api.github.com/users/jnishi/gists{/gist_id}
https://github.com/jnishi
jnishi
MDQ6VXNlcjgzNjU0MQ==
https://api.github.com/users/jnishi/orgs
https://api.github.com/users/jnishi/received_events
https://api.github.com/users/jnishi/repos
https://api.github.com/users/jnishi/starred{/owner}{/repo}
https://api.github.com/users/jnishi/subscriptions
User
https://api.github.com/users/jnishi
public
MEMBER
It would probably require a bit of tweaking, but you can apply it to a dataset, yes.
This should give you a new dataset with sentence pairs you can train a model on.

You can find the documentation about dataset processing here:
https://huggingface.co/docs/datasets/processing.html#processing-data-with-map
2021-03-30T10:01:58Z
https://github.com/huggingface/datasets/issues/2129#issuecomment-810090649
https://api.github.com/repos/huggingface/datasets/issues/2129
MDEyOklzc3VlQ29tbWVudDgxMDA5MDY0OQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/810090649/reactions
2021-03-30T10:01:58Z
https://api.github.com/repos/huggingface/datasets/issues/comments/810090649
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
Thank you for detail information.

I'll try to apply `create_examples_from_document` to `Dataset` object.

2021-04-01T04:58:40Z
https://github.com/huggingface/datasets/issues/2129#issuecomment-811637925
https://api.github.com/repos/huggingface/datasets/issues/2129
MDEyOklzc3VlQ29tbWVudDgxMTYzNzkyNQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/811637925/reactions
2021-04-01T04:58:40Z
https://api.github.com/repos/huggingface/datasets/issues/comments/811637925
https://avatars.githubusercontent.com/u/836541?v=4
https://api.github.com/users/jnishi/events{/privacy}
https://api.github.com/users/jnishi/followers
https://api.github.com/users/jnishi/following{/other_user}
https://api.github.com/users/jnishi/gists{/gist_id}
https://github.com/jnishi
jnishi
MDQ6VXNlcjgzNjU0MQ==
https://api.github.com/users/jnishi/orgs
https://api.github.com/users/jnishi/received_events
https://api.github.com/users/jnishi/repos
https://api.github.com/users/jnishi/starred{/owner}{/repo}
https://api.github.com/users/jnishi/subscriptions
User
https://api.github.com/users/jnishi
public
Hello.

I'm trying to pretrain the BERT model with next sentence prediction. Is there any function that supports next sentence prediction 
like ` TextDatasetForNextSentencePrediction` of `huggingface/transformers` ?
