Add new SentenceTransformer model

1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+  "word_embedding_dimension": 1024,
+  "pooling_mode_cls_token": true,
+  "pooling_mode_mean_tokens": false,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false,
+  "pooling_mode_weightedmean_tokens": false,
+  "pooling_mode_lasttoken": false,
+  "include_prompt": true
+}

README.md

@@ -0,0 +1,675 @@
+---
+tags:
+- sentence-transformers
+- sentence-similarity
+- feature-extraction
+- generated_from_trainer
+- dataset_size:334
+- loss:MatryoshkaLoss
+- loss:MultipleNegativesRankingLoss
+base_model: Snowflake/snowflake-arctic-embed-l
+widget:
+- source_sentence: 'QUESTION #1: What are the potential adverse effects associated
+    with the use of peramivir?'
+  sentences:
+  - "although poultry-to-human and human-to-human trans -\nmission remains relatively\
+    \ low. Despite low transmissibility, \nthe reported fatality rate is high (approximately\
+    \ 60%).14\nPrevention\nThe Centers for Disease Control and Prevention’s (CDC’s)\
+    \ \nAdvisory Committee on Immunization Practices (ACIP) \nand the American Academy\
+    \ of Family Physicians (AAFP) \nrecommend annual influenza vaccination for all\
+    \ people six \nmonths and older who do not have contraindications. 15,16 \nVaccination\
+    \ efforts should target people at increased risk of \ncomplicated or severe influenza\
+    \ (Table 117-19) and those who \ncare for or live with high-risk individuals,\
+    \ including health \ncare professionals. 15 Two previous FPM articles provided"
+  - 'increased sensitivity to pain.60 These cytokines are also
+
+    associated with URTIs and may mediate mood changes
+
+    associated with these infections.
+
+    Anorexia
+
+    Anorexia is a common behavioural response to URTIs,
+
+    and this response has entered the folklore as advice to
+
+    Figure 4: Fever is caused by cytokines released from macrophages and other
+
+    immune cells
+
+    The cytokines may act on vagal nerve endings or enter the brain to cause a
+
+    resetting of the temperature control centre in the hypothalamus. The
+
+    hypothalamus causes shivering and constriction of skin blood vessels and also
+
+    initiates a sensation of chilliness that is perceived at the level of the cerebral
+
+    cortex. IL=interleukin; TNF=tumour necrosis factor.
+
+    Vagal
+
+    nerves
+
+    ShiveringMacrophages'
+  - "older who have been \nsymptomatic for no \nmore than 48 hours\nContraindicated\
+    \ in people \nwith serious hypersensitivity or \nanaphylaxis to peramivir or any\
+    \ \ncomponent of the product\nPotential adverse effects include \ndiarrhea, nausea,\
+    \ vomiting, and \nneutropenia\nWeigh risks and benefits during \npregnancy; no\
+    \ human data \navailable; no known risk of \nembryo-fetal toxicity based on \n\
+    animal data at 8 times the recom -\nmended human dose; possible \nrisk of embryo-fetal\
+    \ toxicity with \ncontinuous intravenous infusion \nbased on limited animal data\n\
+    Baloxavir (Xofluza), \navailable as oral \ntablets\nNA ($160) Adults and children\
+    \ 12 years \nand older:  \n88 to 174 lb (40 to 79 kg):  \nsingle dose of 40 mg\
+    \  \n≥ 175 lb (80 kg):  single dose \nof 80 mg"
+- source_sentence: Why is Influenza A most responsible for causing pandemics?
+  sentences:
+  - "on the first day of symptoms, medications containing ibu -\nprofen and pseudoephedrine\
+    \ may reduce the severity of cold \nsymptoms.35 Antihistamine monotherapy is not\
+    \ effective \nfor relieving cough.6,23\nIpratropium. Inhaled ipratropium is the\
+    \ only medication \nthat improves persistent cough related to URI in adults. 24,36\
+    \  \nTABLE 1\nDifferential Diagnosis for the Common Cold\nDiagnosis\nSymptom \n\
+    onset Cough Sore throat Fever Rhinorrhea Aches Watery eyes Sneezing\nNasal  \n\
+    congestion Headache\nShortness  \nof breath\nAcute \nbronchitis\nGradual Prominent,\
+    \ per-\nsistent, dry or wet\nCommon None or low \ngrade\nUncommon Mild Common\
+    \ Uncommon Uncommon Common, mild Common\nAllergic \nrhinitis\nGradual Common,\
+    \ chronic Possible, especially \non awakening\nNone Common,"
+  - "Patient information:   Handouts on this topic are available \nat https:// family\
+    \ doctor.org/preventing-the-flu and https:// \nfamily doctor.org/flu-myths.\n\
+    Influenza is an acute viral respiratory infection that causes significant morbidity\
+    \ and mortality worldwide. Three types of influ-\nenza cause disease in humans.\
+    \ Influenza A is the type most responsible for causing pandemics because of its\
+    \ high susceptibility \nto antigenic variation. Influenza is highly contagious,\
+    \ and the hallmark of infection is abrupt onset of fever, cough, chills or \n\
+    sweats, myalgias, and malaise. For most patients in the outpatient setting, the\
+    \ diagnosis is made clinically, and laboratory con-"
+  - "www.aafp.org/fpm/2017/0900/p6.html\n 22.  Centers for Disease Control and Prevention.\
+    \ Influenza (flu):  immuno -\ngenicity, efficacy, and effectiveness of influenza\
+    \ vaccines. Updated \nAugust 23, 2018. Accessed January 22, 2019. https:// www.cdc.gov/flu/\n\
+    professionals/acip/2018-2019/background/immunogenicity.htm\n 23.  DiazGranados\
+    \ CA, Dunning AJ, Kimmel M, et al. Efficacy of high-dose \nversus standard-dose\
+    \ influenza vaccine in older adults. N Engl J Med. \n2014; 371(7): 635-645.\n\
+    \ 24.  DiazGranados CA, Robertson CA, Talbot HK, et al. Prevention of serious\
+    \ \nevents in adults 65 years of age or older:  a comparison between high-\ndose\
+    \ and standard-dose inactivated influenza vaccines. Vaccine. 2015;  \n33(38):\
+    \ 4988-4993."
+- source_sentence: How does the negative likelihood ratio for digital immunoassays
+    compare between adults and children for Influenza A?
+  sentences:
+  - "17.  Erlikh IV, Abraham S, Kondamudi VK. Management of influenza. Am \nFam Physician\
+    \ . 2010;  82(9): 1087-1095. Accessed September  5, 2019. \nhttps:// www.aafp.org/afp/2010/1101/p1087.html\n\
+    \ 18.  Centers for Disease Control and Prevention. Influenza (flu):  for clini\
+    \ -\ncians:  antiviral medication. Updated Decemebr 27, 2018. Accessed \nFebruary\
+    \ 24, 2019. https:// www.cdc.gov/flu/professionals/antivirals/\nsummary-clinicians.htm\n\
+    \ 19.  Centers for Disease Control and Prevention. Influenza (flu):  guide for\
+    \  \nconsidering influenza testing. Updated March 4, 2019. Accessed Octo -\nber\
+    \ 5, 2019. https:// www.cdc.gov/flu/professionals/diagnosis/consider-\ninfluenza-testing.htm"
+  - "TABLE 3\nAccuracy of Point-of-Care Tests for Influenza\nTest\nPositive  \nlikelihood\
+    \  \nratio\nNegative \nlikelihood \nratio\nLow prevalence (5%) High prevalence\
+    \ (33%)\nPositive \npredictive \nvalue (%)\nNegative \npredictive \nvalue (%)\n\
+    Positive \npredictive \nvalue (%)\nNegative \npredictive \nvalue (%)\nInfluenza\
+    \ A\nAdults       \nCommercially available rapid influenza tests 85 0.58 82 3\
+    \ 98 22\nDigital immunoassays 23 0.25 55 1 92 11\nRapid nucleic acid amplification\
+    \ tests 44 0.13 70 1 96 6\nChildren       \nCommercially available rapid influenza\
+    \ tests 76 0.39 80 2 97 16\nDigital immunoassays 46 0.13 71 1 96 6\nRapid nucleic\
+    \ acid amplification tests 90 0.10 83 0 98 5\nInfluenza B\nAdults       \nCommercially\
+    \ available rapid influenza tests 332 0.67 95 3 99 25"
+  - "recommended dosages. 28 However, extended treatment \ncourses may be indicated\
+    \ in critically ill patients. 18 Support-\nive treatment and management of complications,\
+    \ including \npotential secondary bacterial pneumonia, are paramount. \nCorticosteroids\
+    \ are not recommended unless the patient \nhas another approved indication for\
+    \ their use.18,28 Treatment \nresistance should be considered in patients who\
+    \ take anti -\nvirals and develop lower respiratory tract disease, although \n\
+    this is less likely than natural disease progression and more \ncommon in immunosuppressed\
+    \ patients.18\nPregnancy is an independent risk factor for complicated \ninfluenza.\
+    \ The risk of maternal death increases with each"
+- source_sentence: What is the role of ipratropium in the treatment of the common
+    cold according to the context?
+  sentences:
+  - "sistent, dry or wet\nCommon None or low \ngrade\nUncommon Mild Common Uncommon\
+    \ Uncommon Common, mild Common\nAllergic \nrhinitis\nGradual Common, chronic Possible,\
+    \ especially \non awakening\nNone Common, \nprominent\nNone Common Prominent Common\
+    \ Uncommon Uncommon\nBacterial \nsinusitis\nGradual Common Common Common Common\
+    \ Common Uncommon Uncommon Common Common Uncommon\nCommon \ncold\nGradual Common,\
+    \ dry Common None or low \ngrade\nCommon Mild Common Common Common Common, mild\
+    \ Uncommon\nInfluenza Abrupt Common, dry \nhacking\nCommon Characteristic;   \
+    \ \nhigh and rises \nrapidly\nCommon Early, \nprominent\nUncommon Uncommon Possible\
+    \ Prominent Uncommon\nPertussis Gradual Prominent, parox-\nysmal, whoop-like\n\
+    Uncommon None or low \ngrade"
+  - 'common cold are inhibited by intranasal administration
+
+    of ipratropium.25 The nasal discharge also consists of a
+
+    protein-rich plasma exudate derived from subepithelial
+
+    capillaries,28 which may explain why anticholinergics
+
+    only partly inhibit nasal discharge associated with
+
+    URTIs.27
+
+    The colour of nasal discharge and sputum is often
+
+    used as a clinical marker to determine whether or not to
+
+    prescribe antibiotics but there is no evidence from the
+
+    literature that supports this concept,29 since colour
+
+    changes in nasal discharge or sputum reflect the severity
+
+    of the inflammatory response30 rather than the nature of
+
+    the infection. Much of the literature relates to colour
+
+    changes in sputum and the lower airways but the same'
+  - "release by leukocytic pyrogen (interleukin-1). A mechanism for the\nincreased\
+    \ degradation of muscle proteins during fever. N Engl J\nMed1983; 308: 553–58.\n\
+    64 Kotler DP. Cachexia. Ann Intern Med2000; 133: 622–34. \n65 Ferreira SH. Prostaglandins,\
+    \ pain, and inflammation. Agents\nActions Suppl1986; 19: 91–98."
+- source_sentence: 'QUESTION #1: How might changes in posture from sitting to supine
+    affect sinus pain according to the context?'
+  sentences:
+  - 'gas absorption from the sinus and “vacuum maxillary
+
+    sinusitis”.37 However, sinuses with patent ostia may also
+
+    be painful, indicating that the generation of
+
+    inflammatory mediators within the sinus may be
+
+    sufficient to trigger the sensation of pain either by direct
+
+    stimulation of pain nerve fibres or via distension of blood
+
+    vessels that are also served by sensory nerves.36 Changes
+
+    in posture from sitting to supine cause an increase in
+
+    sinus pain that may be related to dilation of the blood
+
+    vessels draining the sinus caused by an increase in
+
+    venous pressure. Pressure changes in the sinus may also
+
+    cause pain by stimulation of branches of the trigeminal
+
+    nerve that course in and around the sinuses.37
+
+    Watery eyes'
+  - "American Indians and Alaska Natives\nChildren younger than 5 years (particularly\
+    \ those younger \nthan 2 years)\nInstitutionalized adults (e.g., residents of\
+    \ nursing homes or \nchronic care facilities)\nPregnant and postpartum women (up\
+    \ to 2 weeks postpartum, \nincluding pregnancy loss)\nAdapted with permission\
+    \ from Erlikh IV, Abraham S, Kondamudi VK. \nManagement of influenza. Am Fam Physician.\
+    \ 2010; 82(9): 1088, with \nadditional information from references 18 and 19."
+  - "sary Antibiotics\nStep Examples\nExplain why \nantibiotics will \nnot help\n\
+    “The common cold is caused by a virus, so antibiot -\nics won’t help.”\n“Antibiotics\
+    \ can’t fight viruses like colds. Taking them \nwon’t do any good this time and\
+    \ may hurt their \nchances of fighting bacterial infections you might \nget in\
+    \ the future.”\nSuggest treat-\nments that might \nhelp\n“You can try honey for\
+    \ your cough, ibuprofen or \nacetaminophen for your muscle aches, and nasal or\
+    \ \noral decongestants with or without an antihistamine \nfor your congestion.”\n\
+    Manage expec-\ntations for length \nof illness\n“Cold viruses can make you feel\
+    \ lousy. Most people \nstart to feel better after about a week, but some -\ntimes\
+    \ the cough can last even longer, especially if \nyou smoke.”"
+pipeline_tag: sentence-similarity
+library_name: sentence-transformers
+metrics:
+- cosine_accuracy@1
+- cosine_accuracy@3
+- cosine_accuracy@5
+- cosine_accuracy@10
+- cosine_precision@1
+- cosine_precision@3
+- cosine_precision@5
+- cosine_precision@10
+- cosine_recall@1
+- cosine_recall@3
+- cosine_recall@5
+- cosine_recall@10
+- cosine_ndcg@10
+- cosine_mrr@10
+- cosine_map@100
+model-index:
+- name: SentenceTransformer based on Snowflake/snowflake-arctic-embed-l
+  results:
+  - task:
+      type: information-retrieval
+      name: Information Retrieval
+    dataset:
+      name: Unknown
+      type: unknown
+    metrics:
+    - type: cosine_accuracy@1
+      value: 0.75
+      name: Cosine Accuracy@1
+    - type: cosine_accuracy@3
+      value: 0.9166666666666666
+      name: Cosine Accuracy@3
+    - type: cosine_accuracy@5
+      value: 1.0
+      name: Cosine Accuracy@5
+    - type: cosine_accuracy@10
+      value: 1.0
+      name: Cosine Accuracy@10
+    - type: cosine_precision@1
+      value: 0.75
+      name: Cosine Precision@1
+    - type: cosine_precision@3
+      value: 0.3055555555555555
+      name: Cosine Precision@3
+    - type: cosine_precision@5
+      value: 0.19999999999999998
+      name: Cosine Precision@5
+    - type: cosine_precision@10
+      value: 0.09999999999999999
+      name: Cosine Precision@10
+    - type: cosine_recall@1
+      value: 0.75
+      name: Cosine Recall@1
+    - type: cosine_recall@3
+      value: 0.9166666666666666
+      name: Cosine Recall@3
+    - type: cosine_recall@5
+      value: 1.0
+      name: Cosine Recall@5
+    - type: cosine_recall@10
+      value: 1.0
+      name: Cosine Recall@10
+    - type: cosine_ndcg@10
+      value: 0.8864909792836682
+      name: Cosine Ndcg@10
+    - type: cosine_mrr@10
+      value: 0.8486111111111113
+      name: Cosine Mrr@10
+    - type: cosine_map@100
+      value: 0.8486111111111111
+      name: Cosine Map@100
+---
+
+# SentenceTransformer based on Snowflake/snowflake-arctic-embed-l
+
+This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [Snowflake/snowflake-arctic-embed-l](https://huggingface.co/Snowflake/snowflake-arctic-embed-l). It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
+
+## Model Details
+
+### Model Description
+- **Model Type:** Sentence Transformer
+- **Base model:** [Snowflake/snowflake-arctic-embed-l](https://huggingface.co/Snowflake/snowflake-arctic-embed-l) <!-- at revision d8fb21ca8d905d2832ee8b96c894d3298964346b -->
+- **Maximum Sequence Length:** 512 tokens
+- **Output Dimensionality:** 1024 dimensions
+- **Similarity Function:** Cosine Similarity
+<!-- - **Training Dataset:** Unknown -->
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
+- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
+
+### Full Model Architecture
+
+```
+SentenceTransformer(
+  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
+  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+  (2): Normalize()
+)
+```
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import SentenceTransformer
+
+# Download from the 🤗 Hub
+model = SentenceTransformer("Gonalb/flucold-ft-v0")
+# Run inference
+sentences = [
+    'QUESTION #1: How might changes in posture from sitting to supine affect sinus pain according to the context?',
+    'gas absorption from the sinus and “vacuum maxillary\nsinusitis”.37 However, sinuses with patent ostia may also\nbe painful, indicating that the generation of\ninflammatory mediators within the sinus may be\nsufficient to trigger the sensation of pain either by direct\nstimulation of pain nerve fibres or via distension of blood\nvessels that are also served by sensory nerves.36 Changes\nin posture from sitting to supine cause an increase in\nsinus pain that may be related to dilation of the blood\nvessels draining the sinus caused by an increase in\nvenous pressure. Pressure changes in the sinus may also\ncause pain by stimulation of branches of the trigeminal\nnerve that course in and around the sinuses.37\nWatery eyes',
+    'American Indians and Alaska Natives\nChildren younger than 5 years (particularly those younger \nthan 2 years)\nInstitutionalized adults (e.g., residents of nursing homes or \nchronic care facilities)\nPregnant and postpartum women (up to 2 weeks postpartum, \nincluding pregnancy loss)\nAdapted with permission from Erlikh IV, Abraham S, Kondamudi VK. \nManagement of influenza. Am Fam Physician. 2010; 82(9): 1088, with \nadditional information from references 18 and 19.',
+]
+embeddings = model.encode(sentences)
+print(embeddings.shape)
+# [3, 1024]
+
+# Get the similarity scores for the embeddings
+similarities = model.similarity(embeddings, embeddings)
+print(similarities.shape)
+# [3, 3]
+```
+
+<!--
+### Direct Usage (Transformers)
+
+<details><summary>Click to see the direct usage in Transformers</summary>
+
+</details>
+-->
+
+<!--
+### Downstream Usage (Sentence Transformers)
+
+You can finetune this model on your own dataset.
+
+<details><summary>Click to expand</summary>
+
+</details>
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+## Evaluation
+
+### Metrics
+
+#### Information Retrieval
+
+* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
+
+| Metric              | Value      |
+|:--------------------|:-----------|
+| cosine_accuracy@1   | 0.75       |
+| cosine_accuracy@3   | 0.9167     |
+| cosine_accuracy@5   | 1.0        |
+| cosine_accuracy@10  | 1.0        |
+| cosine_precision@1  | 0.75       |
+| cosine_precision@3  | 0.3056     |
+| cosine_precision@5  | 0.2        |
+| cosine_precision@10 | 0.1        |
+| cosine_recall@1     | 0.75       |
+| cosine_recall@3     | 0.9167     |
+| cosine_recall@5     | 1.0        |
+| cosine_recall@10    | 1.0        |
+| **cosine_ndcg@10**  | **0.8865** |
+| cosine_mrr@10       | 0.8486     |
+| cosine_map@100      | 0.8486     |
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Dataset
+
+#### Unnamed Dataset
+
+* Size: 334 training samples
+* Columns: <code>sentence_0</code> and <code>sentence_1</code>
+* Approximate statistics based on the first 334 samples:
+  |         | sentence_0                                                                         | sentence_1                                                                           |
+  |:--------|:-----------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|
+  | type    | string                                                                             | string                                                                               |
+  | details | <ul><li>min: 12 tokens</li><li>mean: 24.85 tokens</li><li>max: 61 tokens</li></ul> | <ul><li>min: 61 tokens</li><li>mean: 159.74 tokens</li><li>max: 248 tokens</li></ul> |
+* Samples:
+  | sentence_0                                                                                                                    | sentence_1                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
+  |:------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>QUESTION #1: What is the source website from which the document was downloaded?</code>                                  | <code>Downloaded from the American Family Physician website at www.aafp.org/afp. Copyright © 2019 American Academy of Family Physicians. For the private, noncom -<br>mercial use of one individual user of the website. All other rights reserved. Contact [email protected] for copyright questions and/or permission requests.</code>                                                                                                                                                                                                                                                                                                                                                                                                                                           |
+  | <code>QUESTION #2: Who should be contacted for copyright questions and/or permission requests regarding the document?</code>  | <code>Downloaded from the American Family Physician website at www.aafp.org/afp. Copyright © 2019 American Academy of Family Physicians. For the private, noncom -<br>mercial use of one individual user of the website. All other rights reserved. Contact [email protected] for copyright questions and/or permission requests.</code>                                                                                                                                                                                                                                                                                                                                                                                                                                           |
+  | <code>QUESTION #1: Why is early diagnosis essential for antiviral therapy and public-health measures in the community?</code> | <code>syndrome (SARS) 3 because early diagnosis is essential<br>for any antiviral therapy and for the initiation of public-<br>health measures in the community (eg, isolation of<br>infected cases). Here, I discuss the mechanisms that<br>generate symptoms associated with URTIs, especially<br>common cold and flu, but will not review virology in any<br>detail except as regards relevance to symptoms. <br>Is it a cold or flu?<br>The clinical expression of URTIs is variable and is<br>partly influenced by the nature of the infecting virus<br>but to a greater extent is modulated by the age,<br>physiological state, and immunological experience of<br>the host. 4 Depending on these factors, URTIs may<br>occur without symptoms, may kill, or most commonly</code> |
+* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
+  ```json
+  {
+      "loss": "MultipleNegativesRankingLoss",
+      "matryoshka_dims": [
+          768,
+          512,
+          256,
+          128,
+          64
+      ],
+      "matryoshka_weights": [
+          1,
+          1,
+          1,
+          1,
+          1
+      ],
+      "n_dims_per_step": -1
+  }
+  ```
+
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+
+- `eval_strategy`: steps
+- `per_device_train_batch_size`: 10
+- `per_device_eval_batch_size`: 10
+- `num_train_epochs`: 10
+- `multi_dataset_batch_sampler`: round_robin
+
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+
+- `overwrite_output_dir`: False
+- `do_predict`: False
+- `eval_strategy`: steps
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 10
+- `per_device_eval_batch_size`: 10
+- `per_gpu_train_batch_size`: None
+- `per_gpu_eval_batch_size`: None
+- `gradient_accumulation_steps`: 1
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 5e-05
+- `weight_decay`: 0.0
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1
+- `num_train_epochs`: 10
+- `max_steps`: -1
+- `lr_scheduler_type`: linear
+- `lr_scheduler_kwargs`: {}
+- `warmup_ratio`: 0.0
+- `warmup_steps`: 0
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `save_safetensors`: True
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `no_cuda`: False
+- `use_cpu`: False
+- `use_mps_device`: False
+- `seed`: 42
+- `data_seed`: None
+- `jit_mode_eval`: False
+- `use_ipex`: False
+- `bf16`: False
+- `fp16`: False
+- `fp16_opt_level`: O1
+- `half_precision_backend`: auto
+- `bf16_full_eval`: False
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: 0
+- `ddp_backend`: None
+- `tpu_num_cores`: None
+- `tpu_metrics_debug`: False
+- `debug`: []
+- `dataloader_drop_last`: False
+- `dataloader_num_workers`: 0
+- `dataloader_prefetch_factor`: None
+- `past_index`: -1
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: False
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_min_num_params`: 0
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `fsdp_transformer_layer_cls_to_wrap`: None
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch
+- `optim_args`: None
+- `adafactor`: False
+- `group_by_length`: False
+- `length_column_name`: length
+- `ddp_find_unused_parameters`: None
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `use_legacy_prediction_loop`: False
+- `push_to_hub`: False
+- `resume_from_checkpoint`: None
+- `hub_model_id`: None
+- `hub_strategy`: every_save
+- `hub_private_repo`: None
+- `hub_always_push`: False
+- `gradient_checkpointing`: False
+- `gradient_checkpointing_kwargs`: None
+- `include_inputs_for_metrics`: False
+- `include_for_metrics`: []
+- `eval_do_concat_batches`: True
+- `fp16_backend`: auto
+- `push_to_hub_model_id`: None
+- `push_to_hub_organization`: None
+- `mp_parameters`: 
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `torchdynamo`: None
+- `ray_scope`: last
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `dispatch_batches`: None
+- `split_batches`: None
+- `include_tokens_per_second`: False
+- `include_num_input_tokens_seen`: False
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `use_liger_kernel`: False
+- `eval_use_gather_object`: False
+- `average_tokens_across_devices`: False
+- `prompts`: None
+- `batch_sampler`: batch_sampler
+- `multi_dataset_batch_sampler`: round_robin
+
+</details>
+
+### Training Logs
+| Epoch  | Step | cosine_ndcg@10 |
+|:------:|:----:|:--------------:|
+| 1.0    | 34   | 0.9108         |
+| 1.4706 | 50   | 0.9098         |
+| 2.0    | 68   | 0.8834         |
+| 2.9412 | 100  | 0.9051         |
+| 3.0    | 102  | 0.9066         |
+| 4.0    | 136  | 0.9205         |
+| 4.4118 | 150  | 0.9019         |
+| 5.0    | 170  | 0.9156         |
+| 5.8824 | 200  | 0.9247         |
+| 6.0    | 204  | 0.9238         |
+| 7.0    | 238  | 0.9019         |
+| 7.3529 | 250  | 0.8856         |
+| 8.0    | 272  | 0.8856         |
+| 8.8235 | 300  | 0.8879         |
+| 9.0    | 306  | 0.8879         |
+| 10.0   | 340  | 0.8865         |
+
+
+### Framework Versions
+- Python: 3.11.11
+- Sentence Transformers: 3.4.1
+- Transformers: 4.48.3
+- PyTorch: 2.5.1+cu124
+- Accelerate: 1.3.0
+- Datasets: 3.3.2
+- Tokenizers: 0.21.0
+
+## Citation
+
+### BibTeX
+
+#### Sentence Transformers
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
+    author = "Reimers, Nils and Gurevych, Iryna",
+    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
+    month = "11",
+    year = "2019",
+    publisher = "Association for Computational Linguistics",
+    url = "https://arxiv.org/abs/1908.10084",
+}
+```
+
+#### MatryoshkaLoss
+```bibtex
+@misc{kusupati2024matryoshka,
+    title={Matryoshka Representation Learning},
+    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
+    year={2024},
+    eprint={2205.13147},
+    archivePrefix={arXiv},
+    primaryClass={cs.LG}
+}
+```
+
+#### MultipleNegativesRankingLoss
+```bibtex
+@misc{henderson2017efficient,
+    title={Efficient Natural Language Response Suggestion for Smart Reply},
+    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
+    year={2017},
+    eprint={1705.00652},
+    archivePrefix={arXiv},
+    primaryClass={cs.CL}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,25 @@
+{
+  "_name_or_path": "Snowflake/snowflake-arctic-embed-l",
+  "architectures": [
+    "BertModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 1024,
+  "initializer_range": 0.02,
+  "intermediate_size": 4096,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 24,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float32",
+  "transformers_version": "4.48.3",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

config_sentence_transformers.json

@@ -0,0 +1,12 @@
+{
+  "__version__": {
+    "sentence_transformers": "3.4.1",
+    "transformers": "4.48.3",
+    "pytorch": "2.5.1+cu124"
+  },
+  "prompts": {
+    "query": "Represent this sentence for searching relevant passages: "
+  },
+  "default_prompt_name": null,
+  "similarity_fn_name": "cosine"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fac91b1318a851d3ea7340b12e2d7b42a384f6d1dbd763a59717bcfb5c16b944
+size 1336413848

modules.json

@@ -0,0 +1,20 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  },
+  {
+    "idx": 2,
+    "name": "2",
+    "path": "2_Normalize",
+    "type": "sentence_transformers.models.Normalize"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 512,
+  "do_lower_case": false
+}

special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,63 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_lower_case": true,
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "max_length": 512,
+  "model_max_length": 512,
+  "pad_to_multiple_of": null,
+  "pad_token": "[PAD]",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "sep_token": "[SEP]",
+  "stride": 0,
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]"
+}