Add new SentenceTransformer model

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+  "word_embedding_dimension": 768,
+  "pooling_mode_cls_token": false,
+  "pooling_mode_mean_tokens": true,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false,
+  "pooling_mode_weightedmean_tokens": false,
+  "pooling_mode_lasttoken": false,
+  "include_prompt": true
+}

README.md

@@ -0,0 +1,578 @@
+---
+tags:
+- sentence-transformers
+- sentence-similarity
+- feature-extraction
+- generated_from_trainer
+- dataset_size:65360
+- loss:ContrastiveLoss
+base_model: BSC-LT/mRoBERTa
+widget:
+- source_sentence: El agua se volvió turbia desde que uso el ozono. ¿A qué se debe?
+  sentences:
+  - '27 de septiembre de 2009 27 de septiembre de 2009Tegucigalpa, Honduras — En un
+    comunicado del gobierno de facto hondureño, encabezado por Roberto Micheletti,
+    se ha decretado un ultimatum de 10 días para que la embajada de Brasil defina
+    el estatus del depuesto presidente Manuel Zelaya, quien permanece en la sede diplomática
+    desde el lunes pasado. En el texto del comunicado se afirma: "ningún país puede
+    tolerar que una embajada extranjera sea utilizada como base de mando para generar
+    violencia y romper la tranquilidad, como el señor Zelaya lo ha estado haciendo
+    desde su ingreso al territorio nacional", agregando: "nos veremos obligados a
+    tomar medidas adicionales conforme al derecho internacional".'
+  - La nubosidad es una señal de que el ozono está haciendo su trabajo. Continúe ejecutando
+    el filtro y desaparecerá en un par de días. La nubosidad también puede indicar
+    un problema en el filtro. Verifique el filtro por daños y operación apropiada.
+    Retrolavado si es necesario.
+  - El filòsof i periodista Jordi Graupera ha guanyat la votació de Primàries Catalunya
+    a Barcelona, amb 7.715 vots. El segon candidat més votat ha estat Adrià Alsina,
+    ex-membre del secretariat de l'ANC, amb 1.933 vots. La votació va començar divendres
+    a les 20.00 i s'ha acabat avui a les 20.00.
+- source_sentence: What is a term being used for organizations that are for the citizens,
+    by the citizens?
+  sentences:
+  - 'The term "civil society organization" (CSO) has been used by a growing number
+    of organizations, such as the Center for the Study of Global Governance. The term
+    "citizen sector organization" (CSO) has also been advocated to describe the sector
+    – as one of citizens, for citizens – by organizations such as Ashoka: Innovators
+    for the Public. A more broadly applicable term, "Social Benefit Organization"
+    (SBO) has been advocated for by organizations such as MiniDonations.'
+  - The number e is a famous irrational number and is one of the most important numbers
+    in mathematics. e is the base of the Natural Logarithms.
+  - In 1885, British claims to a West African sphere of influence received recognition
+    from other European nations at the Berlin Conference. The following year, it chartered
+    the Royal Niger Company under the leadership of Sir George Taubman Goldie.
+- source_sentence: ¿Cuándo se llevó a cabo la adquisición?
+  sentences:
+  - La operadora de telefonía móvil de Sao Paulo, Telesp Celular, controlada por Portugal
+    Telecom, adquirió el control de la empresa brasileña Ceterp Celular, perteneciente
+    al grupo español Telefónica, se informó hoy. En un aviso publicitario publicado
+    hoy en la prensa local, Telefónica indica que la operación se hizo efectiva ayer,
+    Martes, por un valor de 148,5 millones de reales (75,5 millones de dólares). La
+    empresa vendida formaba parte de las Centrales Telefónicas de Ribeirao Preto (Ceterp),
+    una pequeña compañía comprada en diciembre del año pasado por Telefónica en subasta
+    pública.
+  - Los ecologistas denuncian ante el TSJC la concesión por la Agència Catalana del'
+    Aigua de caudales de la depuradora de Les Fonts para el futuro campo de Torrebonica.
+    Adenc, que ya denunció el uso del agua, cree que han existido irregularidades
+    en la concesión.
+  - The word Anglican originates in ecclesia anglicana, a medieval Latin phrase dating
+    to at least 1246 that means the English Church. There is no single "Anglican Church"
+    with universal juridical authority, since each national or regional church has
+    full autonomy. As the name suggests, the communion is an association of churches
+    in full communion with the Archbishop of Canterbury.
+- source_sentence: What was introduced that made using Mac OS alternative operating
+    systems easier?
+  sentences:
+  - and then to hand over control to a Mac OS-based bootloader application. Used even
+    by Apple for A/UX and MkLinux, this technique is no longer necessary since the
+    introduction of Open Firmware-based PCI Macs, though it was formerly used for
+    convenience on many Old World ROM systems due to bugs in the firmware implementation.[citation
+    needed] Now, Mac hardware boots directly from Open Firmware in most PowerPC-based
+    Macs or EFI in all Intel-based Macs.
+  - During the Napoleonic Wars in the late 18th century and early 19th century, Napoleon
+    annexed territory formerly controlled by the Habsburgs and Savoys. In 1798 he
+    established the Helvetic Republic in Switzerland; two years later he led an army
+    across the St. Bernard pass and conquered almost all of the Alpine regions.
+  - La visita coincideix amb un període de vaga a França per la reforma de les pensions
+    que també afecta el col·lectiu que exerceix al Principat. La visita de Jean–Michel
+    Blanquer a Andorra ha arrencat a primera hora amb una reunió amb el cap de govern,
+    Xavier Espot. No se'n sap el contingut, però de segur que han parlat del reforç
+    de la presència francesa a Andorra mitjançant el sistema educatiu.
+- source_sentence: Of pregnant women, how many are believed to be infected with HIV?
+  sentences:
+  - In 2004, the Swaziland government acknowledged for the first time that it suffered
+    an AIDS crisis, with 38.8% of tested pregnant women infected with HIV (see AIDS
+    in Africa). The then Prime Minister Themba Dlamini declared a humanitarian crisis
+    due to the combined effect of drought, land degradation, increased poverty, and
+    HIV/AIDS.
+  - James Liebman, a professor of law at Columbia Law School, stated in 1996 that
+    his study found that when habeas corpus petitions in death penalty cases were
+    traced from conviction to completion of the case that there was "a 40 percent
+    success rate in all capital cases from 1978 to 1995." Similarly, a study by Ronald
+    Tabak in a law review article puts the success rate in habeas corpus cases involving
+    death row inmates even higher, finding that between "1976 and 1991, approximately
+    47 percent of the habeas petitions filed by death row inmates were granted."
+  - Si t'allotges a Casa da Pendôa tens aquestes possibilitats d'aparcament (sota
+    disponibilitat):Pàrquing interior de pagament,Pàrquing exterior de pagament
+datasets:
+- langtech-innovation/trilingual_query_relevance
+pipeline_tag: sentence-similarity
+library_name: sentence-transformers
+metrics:
+- cosine_accuracy
+- cosine_accuracy_threshold
+- cosine_f1
+- cosine_f1_threshold
+- cosine_precision
+- cosine_recall
+- cosine_ap
+- cosine_mcc
+model-index:
+- name: SentenceTransformer based on BSC-LT/mRoBERTa
+  results:
+  - task:
+      type: binary-classification
+      name: Binary Classification
+    dataset:
+      name: trilingual query relevance dev
+      type: trilingual_query_relevance_dev
+    metrics:
+    - type: cosine_accuracy
+      value: 0.9286237635852973
+      name: Cosine Accuracy
+    - type: cosine_accuracy_threshold
+      value: 0.7209097743034363
+      name: Cosine Accuracy Threshold
+    - type: cosine_f1
+      value: 0.9293707135122127
+      name: Cosine F1
+    - type: cosine_f1_threshold
+      value: 0.7056418657302856
+      name: Cosine F1 Threshold
+    - type: cosine_precision
+      value: 0.9159255306533856
+      name: Cosine Precision
+    - type: cosine_recall
+      value: 0.9432165099523752
+      name: Cosine Recall
+    - type: cosine_ap
+      value: 0.9810741730295642
+      name: Cosine Ap
+    - type: cosine_mcc
+      value: 0.8570174689645386
+      name: Cosine Mcc
+---
+
+# SentenceTransformer based on BSC-LT/mRoBERTa
+
+This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [BSC-LT/mRoBERTa](https://huggingface.co/BSC-LT/mRoBERTa) on the [trilingual_query_relevance](https://huggingface.co/datasets/crodri/trilingual_query_relevance) dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
+
+## Model Details
+
+### Model Description
+- **Model Type:** Sentence Transformer
+- **Base model:** [BSC-LT/mRoBERTa](https://huggingface.co/BSC-LT/mRoBERTa) <!-- at revision e2d3bb2d12e3a03baa32f1d7d1b685aea0614b86 -->
+- **Maximum Sequence Length:** 512 tokens
+- **Output Dimensionality:** 768 dimensions
+- **Similarity Function:** Cosine Similarity
+- **Training Dataset:**
+    - [trilingual_query_relevance](https://huggingface.co/datasets/crodri/trilingual_query_relevance)
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
+- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
+
+### Full Model Architecture
+
+```
+SentenceTransformer(
+  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: RobertaModel 
+  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+)
+```
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import SentenceTransformer
+
+# Download from the 🤗 Hub
+model = SentenceTransformer("langtech-innovation/mRoBERTA_retrieval")
+# Run inference
+sentences = [
+    'Of pregnant women, how many are believed to be infected with HIV?',
+    'In 2004, the Swaziland government acknowledged for the first time that it suffered an AIDS crisis, with 38.8% of tested pregnant women infected with HIV (see AIDS in Africa). The then Prime Minister Themba Dlamini declared a humanitarian crisis due to the combined effect of drought, land degradation, increased poverty, and HIV/AIDS.',
+    'James Liebman, a professor of law at Columbia Law School, stated in 1996 that his study found that when habeas corpus petitions in death penalty cases were traced from conviction to completion of the case that there was "a 40 percent success rate in all capital cases from 1978 to 1995." Similarly, a study by Ronald Tabak in a law review article puts the success rate in habeas corpus cases involving death row inmates even higher, finding that between "1976 and 1991, approximately 47 percent of the habeas petitions filed by death row inmates were granted."',
+]
+embeddings = model.encode(sentences)
+print(embeddings.shape)
+# [3, 768]
+
+# Get the similarity scores for the embeddings
+similarities = model.similarity(embeddings, embeddings)
+print(similarities.shape)
+# [3, 3]
+```
+
+<!--
+### Direct Usage (Transformers)
+
+<details><summary>Click to see the direct usage in Transformers</summary>
+
+</details>
+-->
+
+<!--
+### Downstream Usage (Sentence Transformers)
+
+You can finetune this model on your own dataset.
+
+<details><summary>Click to expand</summary>
+
+</details>
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+## Evaluation
+
+### Metrics
+
+#### Binary Classification
+
+* Dataset: `trilingual_query_relevance_dev`
+* Evaluated with [<code>BinaryClassificationEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.BinaryClassificationEvaluator)
+
+| Metric                    | Value      |
+|:--------------------------|:-----------|
+| cosine_accuracy           | 0.9286     |
+| cosine_accuracy_threshold | 0.7209     |
+| cosine_f1                 | 0.9294     |
+| cosine_f1_threshold       | 0.7056     |
+| cosine_precision          | 0.9159     |
+| cosine_recall             | 0.9432     |
+| **cosine_ap**             | **0.9811** |
+| cosine_mcc                | 0.857      |
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Dataset
+
+#### trilingual_query_relevance
+
+* Dataset: [trilingual_query_relevance](https://huggingface.co/datasets/crodri/trilingual_query_relevance) at [64c425f](https://huggingface.co/datasets/crodri/trilingual_query_relevance/tree/64c425f05b56dd8ce5002c530fa9887953949ccc)
+* Size: 65,360 training samples
+* Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>label</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | sentence1                                                                          | sentence2                                                                           | label                        |
+  |:--------|:-----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:-----------------------------|
+  | type    | string                                                                             | string                                                                              | int                          |
+  | details | <ul><li>min: 3 tokens</li><li>mean: 14.05 tokens</li><li>max: 138 tokens</li></ul> | <ul><li>min: 10 tokens</li><li>mean: 92.67 tokens</li><li>max: 390 tokens</li></ul> | <ul><li>1: 100.00%</li></ul> |
+* Samples:
+  | sentence1                                                                                  | sentence2                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | label          |
+  |:-------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------|
+  | <code>Com ha dit Juncker que havia sigut Georgieva com a vicepresidenta?</code>            | <code>La vicepresidenta de la Comissió Europea i responsable de Pressupostos i Recursos Humans, Kristalina Georgieva, ha presentat aquest divendres la seva dimissió i deixarà el càrrec a finals d'any per treballar al Banc Mundial. El president de la CE, Jean-Claude Juncker, ha anunciat en un comunicat que el comissari alemany Günther H. Oettinger assumirà la carpeta de Georgieva, a qui ha definit com una 'excel·lent vicepresidenta'. Amb la vacant de Georgieva, s'iniciarà el procés per nomenar un nou comissari búlgar, que haurà de ser avalat pel Parlament Europeu.</code>                                                                                                                                                                                                                                                     | <code>1</code> |
+  | <code>¿Cuándo intentaron agredir a Kiko?</code>                                            | <code>El infierno del Atlético en Segunda División es especialmente duro para Kiko, precisamente el jugador que prestó su imagen a la campaña publicitaria con la que el club madrileño buscó el respaldo de su afición. El más carismático jugador rojiblanco hasta hace unos meses fue objeto el sábado por la noche de un intento de agresión a la salida del estadio, después de la humillante derrota ante el Murcia. Un grupo de los más radicales miembros del Frente Atlético le acusó de ser el principal responsable del descenso y le reprochó con suma dureza no colaborar económicamente con la peña para sufragar sus desplazamientos.</code>                                                                                                                                                                                          | <code>1</code> |
+  | <code>¿Cuándo fue la última vez que pudo celebrarse el desfile en la capital turca?</code> | <code>Tras el infructuoso intento de realizar la marcha del domingo, los organizadores lanzaron un comunicado diciendo que "no estamos asustados, estamos aquí, no cambiaremos (...) Ustedes están asustados, cambiarán y se acostumbrarán". El İstanbul Onur Yürüyüşü, nombre local de la marcha del orgullo gay, fue organizado por primera vez en 2003, atrayendo según los reportes, entre decenas de miles y cien mil personas en 2014, año en que se celebró el último desfile y se toparía con una serie de bloqueos en los tres años siguientes. El año pasado, a los organizadores no se les brindó permiso para hacer la marcha tras los ataques militares que enfrentó Estambul, y en 2015 la marcha fue detenida cuando iba a comenzar, y la policía empleó chorros de agua y gas lacrimógeno para dispersar a los manifestantes.</code> | <code>1</code> |
+* Loss: [<code>ContrastiveLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#contrastiveloss) with these parameters:
+  ```json
+  {
+      "distance_metric": "SiameseDistanceMetric.COSINE_DISTANCE",
+      "margin": 0.5,
+      "size_average": true
+  }
+  ```
+
+### Evaluation Dataset
+
+#### trilingual_query_relevance
+
+* Dataset: [trilingual_query_relevance](https://huggingface.co/datasets/crodri/trilingual_query_relevance) at [64c425f](https://huggingface.co/datasets/crodri/trilingual_query_relevance/tree/64c425f05b56dd8ce5002c530fa9887953949ccc)
+* Size: 16,378 evaluation samples
+* Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>label</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | sentence1                                                                          | sentence2                                                                          | label                        |
+  |:--------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:-----------------------------|
+  | type    | string                                                                             | string                                                                             | int                          |
+  | details | <ul><li>min: 3 tokens</li><li>mean: 14.22 tokens</li><li>max: 117 tokens</li></ul> | <ul><li>min: 9 tokens</li><li>mean: 93.29 tokens</li><li>max: 512 tokens</li></ul> | <ul><li>1: 100.00%</li></ul> |
+* Samples:
+  | sentence1                                                                       | sentence2                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  | label          |
+  |:--------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------|
+  | <code>what type of flowers did Gules use?</code>                                | <code>"Azure, three ships with three masts, rigged and under full sail, the sails, pennants and ensigns Argent, each charged with a cross Gules; on a chief of the second a pale quarterly Azure and Gules, on the 1st and 4th a fleur-de-lis or, on the 2nd and 3rd a leopard or, between two roses Gules seeded Or barbed Vert." The shield had as a crest: "A sphere without a frame, bounded with the Zodiac in bend Or, between two pennants flottant Argent, each charged with a cross Gules, over the sphere the words DEUS INDICAT" (Latin: God Indicates).</code> | <code>1</code> |
+  | <code>What are the best pest control methods for Kuala Lumpur's climate?</code> | <code>The best pest control methods for Kuala Lumpur's climate are using insect baits, traps, and insecticides. It is also important to eliminate sources of food, water, and shelter for pests to prevent them from entering the home. Additionally, sealing off openings and cracks, keeping a clean and clutter-free environment, and regular inspection can help reduce the presence of pests in Kuala Lumpur's climate.</code>                                                                                                                                        | <code>1</code> |
+  | <code>How many housing units were there for the 2010 census?</code>             | <code>The population density was 956.4 inhabitants per square mile (321.9/km²). There were 256,930 housing units at an average density of 375.9 per square mile (145.1/km²).</code>                                                                                                                                                                                                                                                                                                                                                                                        | <code>1</code> |
+* Loss: [<code>ContrastiveLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#contrastiveloss) with these parameters:
+  ```json
+  {
+      "distance_metric": "SiameseDistanceMetric.COSINE_DISTANCE",
+      "margin": 0.5,
+      "size_average": true
+  }
+  ```
+
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+
+- `eval_strategy`: steps
+- `per_device_train_batch_size`: 2
+- `per_device_eval_batch_size`: 2
+- `num_train_epochs`: 4
+- `warmup_ratio`: 0.1
+- `fp16`: True
+- `load_best_model_at_end`: True
+- `gradient_checkpointing`: True
+
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+
+- `overwrite_output_dir`: False
+- `do_predict`: False
+- `eval_strategy`: steps
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 2
+- `per_device_eval_batch_size`: 2
+- `per_gpu_train_batch_size`: None
+- `per_gpu_eval_batch_size`: None
+- `gradient_accumulation_steps`: 1
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 5e-05
+- `weight_decay`: 0.0
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1.0
+- `num_train_epochs`: 4
+- `max_steps`: -1
+- `lr_scheduler_type`: linear
+- `lr_scheduler_kwargs`: {}
+- `warmup_ratio`: 0.1
+- `warmup_steps`: 0
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `save_safetensors`: True
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `no_cuda`: False
+- `use_cpu`: False
+- `use_mps_device`: False
+- `seed`: 42
+- `data_seed`: None
+- `jit_mode_eval`: False
+- `use_ipex`: False
+- `bf16`: False
+- `fp16`: True
+- `fp16_opt_level`: O1
+- `half_precision_backend`: auto
+- `bf16_full_eval`: False
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: 0
+- `ddp_backend`: None
+- `tpu_num_cores`: None
+- `tpu_metrics_debug`: False
+- `debug`: []
+- `dataloader_drop_last`: False
+- `dataloader_num_workers`: 0
+- `dataloader_prefetch_factor`: None
+- `past_index`: -1
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: True
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_min_num_params`: 0
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `fsdp_transformer_layer_cls_to_wrap`: None
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch
+- `optim_args`: None
+- `adafactor`: False
+- `group_by_length`: False
+- `length_column_name`: length
+- `ddp_find_unused_parameters`: None
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `use_legacy_prediction_loop`: False
+- `push_to_hub`: False
+- `resume_from_checkpoint`: None
+- `hub_model_id`: None
+- `hub_strategy`: every_save
+- `hub_private_repo`: None
+- `hub_always_push`: False
+- `gradient_checkpointing`: True
+- `gradient_checkpointing_kwargs`: None
+- `include_inputs_for_metrics`: False
+- `include_for_metrics`: []
+- `eval_do_concat_batches`: True
+- `fp16_backend`: auto
+- `push_to_hub_model_id`: None
+- `push_to_hub_organization`: None
+- `mp_parameters`: 
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `torchdynamo`: None
+- `ray_scope`: last
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `dispatch_batches`: None
+- `split_batches`: None
+- `include_tokens_per_second`: False
+- `include_num_input_tokens_seen`: False
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `use_liger_kernel`: False
+- `eval_use_gather_object`: False
+- `average_tokens_across_devices`: False
+- `prompts`: None
+- `batch_sampler`: batch_sampler
+- `multi_dataset_batch_sampler`: proportional
+
+</details>
+
+### Training Logs
+| Epoch  | Step | Training Loss | Validation Loss | trilingual_query_relevance_dev_cosine_ap |
+|:------:|:----:|:-------------:|:---------------:|:----------------------------------------:|
+| 0.0031 | 100  | 0.024         | 0.0241          | 0.8502                                   |
+| 0.0061 | 200  | 0.024         | 0.0232          | 0.8688                                   |
+| 0.0092 | 300  | 0.0238        | 0.0215          | 0.9041                                   |
+| 0.0122 | 400  | 0.022         | 0.0189          | 0.9431                                   |
+| 0.0153 | 500  | 0.0201        | 0.0164          | 0.9610                                   |
+| 0.0184 | 600  | 0.0167        | 0.0145          | 0.9676                                   |
+| 0.0214 | 700  | 0.0143        | 0.0133          | 0.9711                                   |
+| 0.0245 | 800  | 0.0141        | 0.0123          | 0.9689                                   |
+| 0.0275 | 900  | 0.0121        | 0.0117          | 0.9707                                   |
+| 0.0306 | 1000 | 0.0114        | 0.0116          | 0.9704                                   |
+| 0.0337 | 1100 | 0.0121        | 0.0116          | 0.9690                                   |
+| 0.0367 | 1200 | 0.0117        | 0.0112          | 0.9710                                   |
+| 0.0398 | 1300 | 0.0114        | 0.0110          | 0.9690                                   |
+| 0.0428 | 1400 | 0.0124        | 0.0106          | 0.9688                                   |
+| 0.0459 | 1500 | 0.0113        | 0.0104          | 0.9712                                   |
+| 0.0490 | 1600 | 0.0113        | 0.0103          | 0.9740                                   |
+| 0.0520 | 1700 | 0.0114        | 0.0108          | 0.9714                                   |
+| 0.0551 | 1800 | 0.0119        | 0.0105          | 0.9757                                   |
+| 0.0581 | 1900 | 0.011         | 0.0101          | 0.9749                                   |
+| 0.0612 | 2000 | 0.0138        | 0.0097          | 0.9776                                   |
+| 0.0643 | 2100 | 0.0124        | 0.0097          | 0.9775                                   |
+| 0.0673 | 2200 | 0.0109        | 0.0097          | 0.9776                                   |
+| 0.0704 | 2300 | 0.0128        | 0.0106          | 0.9774                                   |
+| 0.0734 | 2400 | 0.0143        | 0.0097          | 0.9811                                   |
+| 0.0765 | 2500 | 0.0125        | 0.0096          | 0.9791                                   |
+| 0.0796 | 2600 | 0.0121        | 0.0113          | 0.9806                                   |
+| 0.0826 | 2700 | 0.009         | 0.0093          | 0.9799                                   |
+| 0.0857 | 2800 | 0.0092        | 0.0108          | 0.9772                                   |
+| 0.0887 | 2900 | 0.0115        | 0.0095          | 0.9760                                   |
+| 0.0918 | 3000 | 0.0115        | 0.0098          | 0.9796                                   |
+| 0.0949 | 3100 | 0.0109        | 0.0092          | 0.9796                                   |
+| 0.0979 | 3200 | 0.0113        | 0.0095          | 0.9753                                   |
+| 0.1010 | 3300 | 0.0095        | 0.0094          | 0.9759                                   |
+| 0.1040 | 3400 | 0.0087        | 0.0097          | 0.9767                                   |
+| 0.1071 | 3500 | 0.0101        | 0.0097          | 0.9745                                   |
+| 0.1102 | 3600 | 0.0125        | 0.0105          | 0.9772                                   |
+| 0.1132 | 3700 | 0.0132        | 0.0091          | 0.9798                                   |
+| 0.1163 | 3800 | 0.0081        | 0.0089          | 0.9809                                   |
+| 0.1193 | 3900 | 0.0109        | 0.0095          | 0.9787                                   |
+| 0.1224 | 4000 | 0.0105        | 0.0090          | 0.9789                                   |
+| 0.1255 | 4100 | 0.0089        | 0.0096          | 0.9774                                   |
+| 0.1285 | 4200 | 0.0094        | 0.0095          | 0.9760                                   |
+| 0.1316 | 4300 | 0.0118        | 0.0096          | 0.9806                                   |
+| 0.1346 | 4400 | 0.0104        | 0.0092          | 0.9787                                   |
+| 0.1377 | 4500 | 0.0113        | 0.0094          | 0.9776                                   |
+| 0.1408 | 4600 | 0.0112        | 0.0094          | 0.9761                                   |
+| 0.1438 | 4700 | 0.01          | 0.0093          | 0.9734                                   |
+| 0.1469 | 4800 | 0.0102        | 0.0100          | 0.9724                                   |
+| 0.1499 | 4900 | 0.0123        | 0.0100          | 0.9753                                   |
+| 0.1530 | 5000 | 0.01          | 0.0102          | 0.9794                                   |
+| 0.1561 | 5100 | 0.0093        | 0.0096          | 0.9772                                   |
+| 0.1591 | 5200 | 0.0146        | 0.0096          | 0.9804                                   |
+| 0.1622 | 5300 | 0.0102        | 0.0098          | 0.9762                                   |
+| 0.1652 | 5400 | 0.0118        | 0.0098          | 0.9768                                   |
+| 0.1683 | 5500 | 0.0113        | 0.0090          | 0.9790                                   |
+| 0.1714 | 5600 | 0.0103        | 0.0096          | 0.9762                                   |
+| 0.1744 | 5700 | 0.0107        | 0.0101          | 0.9802                                   |
+| 0.1775 | 5800 | 0.0124        | 0.0098          | 0.9788                                   |
+| 0.1805 | 5900 | 0.0097        | 0.0095          | 0.9785                                   |
+| 0.1836 | 6000 | 0.0101        | 0.0099          | 0.9736                                   |
+| 0.1867 | 6100 | 0.0116        | 0.0099          | 0.9834                                   |
+| 0.1897 | 6200 | 0.0109        | 0.0088          | 0.9811                                   |
+
+
+### Framework Versions
+- Python: 3.10.16
+- Sentence Transformers: 3.4.1
+- Transformers: 4.49.0
+- PyTorch: 2.5.1+cu124
+- Accelerate: 1.2.1
+- Datasets: 3.2.0
+- Tokenizers: 0.21.0
+
+## Citation
+
+### BibTeX
+
+#### Sentence Transformers
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
+    author = "Reimers, Nils and Gurevych, Iryna",
+    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
+    month = "11",
+    year = "2019",
+    publisher = "Association for Computational Linguistics",
+    url = "https://arxiv.org/abs/1908.10084",
+}
+```
+
+#### ContrastiveLoss
+```bibtex
+@inproceedings{hadsell2006dimensionality,
+    author={Hadsell, R. and Chopra, S. and LeCun, Y.},
+    booktitle={2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06)},
+    title={Dimensionality Reduction by Learning an Invariant Mapping},
+    year={2006},
+    volume={2},
+    number={},
+    pages={1735-1742},
+    doi={10.1109/CVPR.2006.100}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,27 @@
+{
+  "_name_or_path": "output/training_trilingual_query_relevance_BSC-LT-mRoBERTa-2025-03-20_21-52-43/checkpoint-6200/",
+  "architectures": [
+    "RobertaModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "classifier_dropout": null,
+  "eos_token_id": 2,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 514,
+  "model_type": "roberta",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 1,
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float32",
+  "transformers_version": "4.49.0",
+  "type_vocab_size": 1,
+  "use_cache": true,
+  "vocab_size": 256000
+}

config_sentence_transformers.json

@@ -0,0 +1,10 @@
+{
+  "__version__": {
+    "sentence_transformers": "3.4.1",
+    "transformers": "4.49.0",
+    "pytorch": "2.5.1+cu124"
+  },
+  "prompts": {},
+  "default_prompt_name": null,
+  "similarity_fn_name": "cosine"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61ece44ba7f8ee74d751061dc315930e9716ac0a0defd0d3fa2ba6c50b118681
+size 1130622968

modules.json

@@ -0,0 +1,14 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 512,
+  "do_lower_case": false
+}

special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9f2bf5621066c920a83c8b0f75b8d0d95b8b1bad04ac245c6746e6d2b303b76b
+size 37007530

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5072e3209a04aa01dbf4db72b8fec52cf8cd06a042c9ba819678e084f7b665d5
+size 4813283

tokenizer_config.json

@@ -0,0 +1,1099 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "add_prefix_space": true,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "5": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "6": {
+      "content": "<|reserved_token_1|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "7": {
+      "content": "<|reserved_token_2|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "8": {
+      "content": "<|reserved_token_3|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "9": {
+      "content": "<|reserved_token_4|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "10": {
+      "content": "<|reserved_token_5|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "11": {
+      "content": "<|reserved_token_6|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "12": {
+      "content": "<|reserved_token_7|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "13": {
+      "content": "<|reserved_token_8|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "14": {
+      "content": "<|reserved_token_9|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "15": {
+      "content": "<|reserved_token_10|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "16": {
+      "content": "<|reserved_token_11|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "17": {
+      "content": "<|reserved_token_12|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "18": {
+      "content": "<|reserved_token_13|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "19": {
+      "content": "<|reserved_token_14|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "20": {
+      "content": "<|reserved_token_15|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "21": {
+      "content": "<|reserved_token_16|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "22": {
+      "content": "<|reserved_token_17|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "23": {
+      "content": "<|reserved_token_18|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "24": {
+      "content": "<|reserved_token_19|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "25": {
+      "content": "<|reserved_token_20|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "26": {
+      "content": "<|reserved_token_21|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "27": {
+      "content": "<|reserved_token_22|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "28": {
+      "content": "<|reserved_token_23|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "29": {
+      "content": "<|reserved_token_24|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "30": {
+      "content": "<|reserved_token_25|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "31": {
+      "content": "<|reserved_token_26|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "32": {
+      "content": "<|reserved_token_27|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "33": {
+      "content": "<|reserved_token_28|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "34": {
+      "content": "<|reserved_token_29|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "35": {
+      "content": "<|reserved_token_30|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "36": {
+      "content": "<|reserved_token_31|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "37": {
+      "content": "<|reserved_token_32|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "38": {
+      "content": "<|reserved_token_33|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "39": {
+      "content": "<|reserved_token_34|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "40": {
+      "content": "<|reserved_token_35|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "41": {
+      "content": "<|reserved_token_36|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "42": {
+      "content": "<|reserved_token_37|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "43": {
+      "content": "<|reserved_token_38|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "44": {
+      "content": "<|reserved_token_39|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "45": {
+      "content": "<|reserved_token_40|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "46": {
+      "content": "<|reserved_token_41|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "47": {
+      "content": "<|reserved_token_42|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "48": {
+      "content": "<|reserved_token_43|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "49": {
+      "content": "<|reserved_token_44|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50": {
+      "content": "<|reserved_token_45|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "51": {
+      "content": "<|reserved_token_46|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "52": {
+      "content": "<|reserved_token_47|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "53": {
+      "content": "<|reserved_token_48|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "54": {
+      "content": "<|reserved_token_49|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "55": {
+      "content": "<|reserved_token_50|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "56": {
+      "content": "<|reserved_token_51|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "57": {
+      "content": "<|reserved_token_52|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "58": {
+      "content": "<|reserved_token_53|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "59": {
+      "content": "<|reserved_token_54|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "60": {
+      "content": "<|reserved_token_55|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "61": {
+      "content": "<|reserved_token_56|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "62": {
+      "content": "<|reserved_token_57|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "63": {
+      "content": "<|reserved_token_58|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "64": {
+      "content": "<|reserved_token_59|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "65": {
+      "content": "<|reserved_token_60|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "66": {
+      "content": "<|reserved_token_61|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "67": {
+      "content": "<|reserved_token_62|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "68": {
+      "content": "<|reserved_token_63|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "69": {
+      "content": "<|reserved_token_64|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "70": {
+      "content": "<|reserved_token_65|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "71": {
+      "content": "<|reserved_token_66|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "72": {
+      "content": "<|reserved_token_67|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "73": {
+      "content": "<|reserved_token_68|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "74": {
+      "content": "<|reserved_token_69|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "75": {
+      "content": "<|reserved_token_70|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "76": {
+      "content": "<|reserved_token_71|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "77": {
+      "content": "<|reserved_token_72|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "78": {
+      "content": "<|reserved_token_73|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "79": {
+      "content": "<|reserved_token_74|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "80": {
+      "content": "<|reserved_token_75|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "81": {
+      "content": "<|reserved_token_76|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "82": {
+      "content": "<|reserved_token_77|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "83": {
+      "content": "<|reserved_token_78|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "84": {
+      "content": "<|reserved_token_79|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "85": {
+      "content": "<|reserved_token_80|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "86": {
+      "content": "<|reserved_token_81|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "87": {
+      "content": "<|reserved_token_82|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "88": {
+      "content": "<|reserved_token_83|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "89": {
+      "content": "<|reserved_token_84|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "90": {
+      "content": "<|reserved_token_85|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "91": {
+      "content": "<|reserved_token_86|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "92": {
+      "content": "<|reserved_token_87|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "93": {
+      "content": "<|reserved_token_88|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "94": {
+      "content": "<|reserved_token_89|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "95": {
+      "content": "<|reserved_token_90|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "96": {
+      "content": "<|reserved_token_91|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "97": {
+      "content": "<|reserved_token_92|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "98": {
+      "content": "<|reserved_token_93|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "99": {
+      "content": "<|reserved_token_94|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "<|reserved_token_95|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "<|reserved_token_96|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "<|reserved_token_97|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "<mask>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "104": {
+      "content": "\\r",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "105": {
+      "content": "▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "106": {
+      "content": "▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "107": {
+      "content": "▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "108": {
+      "content": "▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "109": {
+      "content": "▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "110": {
+      "content": "▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "111": {
+      "content": "▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "112": {
+      "content": "▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "113": {
+      "content": "▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "114": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "115": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "116": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "117": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "118": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "119": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "120": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "121": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "122": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "123": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "124": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "125": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "126": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "127": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "128": {
+      "content": "\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "129": {
+      "content": "\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "130": {
+      "content": "\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "131": {
+      "content": "\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "132": {
+      "content": "\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "133": {
+      "content": "\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "134": {
+      "content": "\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "extra_special_tokens": {},
+  "legacy": true,
+  "model_max_length": 512,
+  "pad_token": "<pad>",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}