YAML Metadata Warning:The pipeline tag "text2text-generation" is not in the official list: text-classification, token-classification, table-question-answering, question-answering, zero-shot-classification, translation, summarization, feature-extraction, text-generation, fill-mask, sentence-similarity, text-to-speech, text-to-audio, automatic-speech-recognition, audio-to-audio, audio-classification, audio-text-to-text, voice-activity-detection, depth-estimation, image-classification, object-detection, image-segmentation, text-to-image, image-to-text, image-to-image, image-to-video, unconditional-image-generation, video-classification, reinforcement-learning, robotics, tabular-classification, tabular-regression, tabular-to-text, table-to-text, multiple-choice, text-ranking, text-retrieval, time-series-forecasting, text-to-video, image-text-to-text, image-text-to-image, image-text-to-video, visual-question-answering, document-question-answering, zero-shot-image-classification, graph-ml, mask-generation, zero-shot-object-detection, text-to-3d, image-to-3d, image-feature-extraction, video-text-to-text, keypoint-detection, visual-document-retrieval, any-to-any, video-to-video, other
wsd-camembert-base-semcor-wngt-fr : almanach/camembert-base fine-tuned on Semcor+WNGT fr for Word Sense Disambiguation
wsd-camembert-base-semcor-wngt-fr is a Word Sense Disambiguation (WSD) model fine-tuned on the French version of Semcor and WNGT datasets with almanach/camembert-base as the pretrained BERT embeddings.
The fine-tuned model achieves the following performance on SemEval 2013 - fr:
| Test F1 (%) | GPUs | Epochs |
|---|---|---|
| 51.28 | 1xV100 32GB | 40 |
📝 Model Details
The WSD model is a Transformer encoder-decoder architecture, consisting of 6 layers in both the encoder and decoder, and leveraging pretrained BERT embeddings for enhanced semantic representation.
💻 How to disambiguate a sentence
To disambiguate a sentence, please refer to the official NWSD repository.
⚙️ Training Details
Training and Test Data
We use Semcor.fr and WNGT.fr annotated with WordNet 3.0 sense keys IDs for the train/valid sets:
| Train | Valid | |
|---|---|---|
| # utterances | 143,597 | 4,000 |
The semeval2013task12.fr.xml test data is the French version of the SemEval-2013 Task 12 test set, with:
| Test | |
|---|---|
| # utterances | 306 |
Training Procedure and Hyperparameters
We follow the training procedure provided in the NWSD github repository.
Training time
With 1xV100 32GB, the training took ~ 4 hours.
Libraries
@inproceedings{vial-etal-2019-sense,
title = "Sense Vocabulary Compression through the Semantic Knowledge of {W}ord{N}et for Neural Word Sense Disambiguation",
author = {Vial, Lo{\"i}c and
Lecouteux, Benjamin and
Schwab, Didier},
editor = "Vossen, Piek and
Fellbaum, Christiane",
booktitle = "Proceedings of the 10th Global Wordnet Conference",
month = jul,
year = "2019",
address = "Wroclaw, Poland",
publisher = "Global Wordnet Association",
url = "https://aclanthology.org/2019.gwc-1.14/",
pages = "108--117",
}
💡 Information
- Developed by: Cécile Macaire
- Funded by [optional]: GENCI-IDRIS (Grant 2023-AD011013625R1) PROPICTO ANR-20-CE93-0005
- Language(s) (NLP): French
- License: MIT
- Finetuned from model: almanach/camembert-base
Model tree for Propicto/wsd-camembert-base-semcor-wngt-fr
Base model
almanach/camembert-base